dreambooth_runpod_joepenna.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "aa2c1ada",
   "metadata": {
    "id": "aa2c1ada"
   },
   "source": [
    "# Dreambooth\n",
    "### Notebook implementation by Joe Penna (@MysteryGuitarM on Twitter) - Improvements by David Bielejeski\n",
    "\n",
    "### Instructions\n",
    "- Sign up for RunPod here: https://runpod.io/?ref=n8yfwyum\n",
    "    - Note: That's my personal referral link. Please don't use it if we are mortal enemies.\n",
    "\n",
    "- Click *Deploy* on either `SECURE CLOUD` or `COMMUNITY CLOUD`\n",
    "\n",
    "- Follow the rest of the instructions in this video: https://www.youtube.com/watch?v=7m__xadX0z0#t=5m33.1s\n",
    "\n",
    "Latest information on:\n",
    "https://github.com/JoePenna/Dreambooth-Stable-Diffusion"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b971cc0",
   "metadata": {
    "id": "7b971cc0"
   },
   "source": [
    "## Build Environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2AsGA1xpNQnb",
   "metadata": {
    "id": "2AsGA1xpNQnb"
   },
   "outputs": [],
   "source": [
    "# If running on Vast.AI, copy the code in this cell into a new notebook. Run it, then launch the `dreambooth_runpod_joepenna.ipynb` notebook from the jupyter interface.\n",
    "!git clone https://github.com/JoePenna/Dreambooth-Stable-Diffusion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9e1bc458-091b-42f4-a125-c3f0df20f29d",
   "metadata": {
    "id": "9e1bc458-091b-42f4-a125-c3f0df20f29d",
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# BUILD ENV\n",
    "!pip install omegaconf\n",
    "!pip install einops\n",
    "!pip install pytorch-lightning==1.6.5\n",
    "!pip install test-tube\n",
    "!pip install transformers\n",
    "!pip install kornia\n",
    "!pip install -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers\n",
    "!pip install -e git+https://github.com/openai/CLIP.git@main#egg=clip\n",
    "!pip install setuptools==59.5.0\n",
    "!pip install pillow==9.0.1\n",
    "!pip install torchmetrics==0.6.0\n",
    "!pip install -e .\n",
    "!pip install protobuf==3.20.1\n",
    "!pip install gdown\n",
    "!pip install -qq diffusers[\"training\"]==0.3.0 transformers ftfy\n",
    "!pip install -qq \"ipywidgets>=7,<8\"\n",
    "!pip install huggingface_hub\n",
    "!pip install ipywidgets==7.7.1\n",
    "!pip install captionizer==1.0.1\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9ebf2e77-1d67-48f7-8958-2e3fe0ab5f38",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip uninstall torch torchvision torchaudio torchtext -y\n",
    "!pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu116\n",
    "!pip install torchtext==0.13.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dae11c10",
   "metadata": {
    "id": "dae11c10"
   },
   "outputs": [],
   "source": [
    "# Hugging Face Login\n",
    "from huggingface_hub import notebook_login\n",
    "\n",
    "notebook_login()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2846bfcc",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "# Download the 1.4 sd model\n",
    "from IPython.display import clear_output\n",
    "\n",
    "from huggingface_hub import hf_hub_download\n",
    "downloaded_model_path = hf_hub_download(\n",
    " repo_id=\"CompVis/stable-diffusion-v-1-4-original\",\n",
    " filename=\"sd-v1-4.ckpt\",\n",
    " use_auth_token=True\n",
    ")\n",
    "\n",
    "# Move the sd-v1-4.ckpt to the root of this directory as \"model.ckpt\"\n",
    "actual_locations_of_model_blob = !readlink -f {downloaded_model_path}\n",
    "!mv {actual_locations_of_model_blob[-1]} model.ckpt\n",
    "clear_output()\n",
    "print(\"✅ model.ckpt successfully downloaded\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17d1d11a",
   "metadata": {
    "id": "17d1d11a"
   },
   "source": [
    "# Regularization Images (Skip this section if you are uploading your own or using the provided images)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed07a5df",
   "metadata": {
    "id": "ed07a5df"
   },
   "source": [
    "Training teaches your new model both your token **but** re-trains your class simultaneously.\n",
    "\n",
    "From cursory testing, it does not seem like reg images affect the model too much. However, they do affect your class greatly, which will in turn affect your generations.\n",
    "\n",
    "You can either generate your images here, or use the repos below to quickly download 1500 images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67f9ff0c-b529-4c7c-8e26-8388d70a5d91",
   "metadata": {
    "id": "67f9ff0c-b529-4c7c-8e26-8388d70a5d91"
   },
   "outputs": [],
   "source": [
    "# GENERATE 200 images - Optional\n",
    "self_generated_files_prompt = \"person\" #@param {type:\"string\"}\n",
    "self_generated_files_count = 200 #@param {type:\"integer\"}\n",
    "\n",
    "!python scripts/stable_txt2img.py \\\n",
    " --seed 10 \\\n",
    " --ddim_eta 0.0 \\\n",
    " --n_samples 1 \\\n",
    " --n_iter {self_generated_files_count} \\\n",
    " --scale 10.0 \\\n",
    " --ddim_steps 50 \\\n",
    " --ckpt model.ckpt \\\n",
    " --prompt {self_generated_files_prompt}\n",
    "\n",
    "dataset=self_generated_files_prompt\n",
    "\n",
    "!mkdir -p regularization_images/{dataset}\n",
    "!mv outputs/txt2img-samples/*.png regularization_images/{dataset}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3d1c7e1c",
   "metadata": {
    "id": "3d1c7e1c"
   },
   "outputs": [],
   "source": [
    "# Zip up the files for downloading and reuse.\n",
    "# Download this file locally so you can reuse during another training on this dataset\n",
    "!apt-get install -y zip\n",
    "!zip -r regularization_images.zip regularization_images/{dataset}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dfb3acd9",
   "metadata": {},
   "source": [
    "# Download pre-generated regularization images\n",
    "We've created the following image sets\n",
    "\n",
    "`man_euler` - provided by Niko Pueringer (Corridor Digital) - euler @ 40 steps, CFG 7.5\n",
    "`man_unsplash` - pictures from various photographers\n",
    "`person_ddim`\n",
    "`woman_ddim` - provided by David Bielejeski - ddim @ 50 steps, CFG 10.0\n",
    "`person_ddim` is recommended"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e7EydXCjOV1v",
   "metadata": {
    "id": "e7EydXCjOV1v",
    "tags": []
   },
   "outputs": [],
   "source": [
    "#Download Regularization Images\n",
    "\n",
    "dataset=\"person_ddim\" #@param [\"man_euler\", \"man_unsplash\", \"person_ddim\", \"woman_ddim\", \"blonde_woman\"]\n",
    "!git clone https://github.com/djbielejeski/Stable-Diffusion-Regularization-Images-{dataset}.git\n",
    "\n",
    "!mkdir -p regularization_images/{dataset}\n",
    "!mv -v Stable-Diffusion-Regularization-Images-{dataset}/{dataset}/*.* regularization_images/{dataset}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8527d40",
   "metadata": {},
   "source": [
    "# Upload your training images\n",
    "Upload 10-20 images of someone to\n",
    "\n",
    "```\n",
    "/workspace/Dreambooth-Stable-Diffusion/training_images\n",
    "```\n",
    "\n",
    "WARNING: Be sure to upload an *even* amount of images, otherwise the training inexplicably stops at 1500 steps.\n",
    "\n",
    "*   2-3 full body\n",
    "*   3-5 upper body\n",
    "*   5-12 close-up on face\n",
    "\n",
    "The images should be:\n",
    "\n",
    "- as close as possible to the kind of images you're trying to make"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2c01baa8",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "#@markdown Add here the URLs to the images of the subject you are adding\n",
    "urls = [\n",
    " \"https://i.imgur.com/test1.png\",\n",
    " \"https://i.imgur.com/test2.png\",\n",
    " \"https://i.imgur.com/test3.png\",\n",
    " \"https://i.imgur.com/test4.png\",\n",
    " \"https://i.imgur.com/test5.png\",\n",
    " # You can add additional images here -- about 20-30 images in different\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ccdb2ee9",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "#@title Download and check the images you have just added\n",
    "import os\n",
    "import requests\n",
    "from io import BytesIO\n",
    "from PIL import Image\n",
    "\n",
    "\n",
    "def image_grid(imgs, rows, cols):\n",
    " assert len(imgs) == rows*cols\n",
    "\n",
    " w, h = imgs[0].size\n",
    " grid = Image.new('RGB', size=(cols*w, rows*h))\n",
    " grid_w, grid_h = grid.size\n",
    "\n",
    " for i, img in enumerate(imgs):\n",
    "  grid.paste(img, box=(i%cols*w, i//cols*h))\n",
    " return grid\n",
    "\n",
    "def download_image(url):\n",
    " try:\n",
    "  response = requests.get(url)\n",
    " except:\n",
    "  return None\n",
    " return Image.open(BytesIO(response.content)).convert(\"RGB\")\n",
    "\n",
    "images = list(filter(None,[download_image(url) for url in urls]))\n",
    "save_path = \"./training_images\"\n",
    "if not os.path.exists(save_path):\n",
    " os.mkdir(save_path)\n",
    "[image.save(f\"{save_path}/{i}.png\", format=\"png\") for i, image in enumerate(images)]\n",
    "image_grid(images, 1, len(images))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad4e50df",
   "metadata": {
    "id": "ad4e50df"
   },
   "source": [
    "## Training\n",
    "\n",
    "If training a person or subject, keep an eye on your project's `logs/{folder}/images/train/samples_scaled_gs-00xxxx` generations.\n",
    "\n",
    "If training a style, keep an eye on your project's `logs/{folder}/images/train/samples_gs-00xxxx` generations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "53220258-0930-4f3c-932d-d7a1ea0ea117",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install gdown\n",
    "!pip install --upgrade --no-cache-dir gdown\n",
    "\n",
    "!gdown https://drive.google.com/uc?id=1X_nZLQlOMY0dsYYjXMS-I-hbfsaFX0gA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6fa5dd66-2ca0-4819-907e-802e25583ae6",
   "metadata": {
    "id": "6fa5dd66-2ca0-4819-907e-802e25583ae6",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Training\n",
    "\n",
    "# This isn't used for training, just to help you remember what your trained into the model.\n",
    "project_name = \"project_name\"\n",
    "\n",
    "# MAX STEPS\n",
    "# How many steps do you want to train for?\n",
    "max_training_steps = 1800\n",
    "\n",
    "# Match class_word to the category of the regularization images you chose above.\n",
    "class_word = \"person\" # typical uses are \"man\", \"person\", \"woman\"\n",
    "\n",
    "# This is the unique token you are incorporating into the stable diffusion model.\n",
    "token = \"freya556\"\n",
    "\n",
    "reg_data_root = \"/workspace/Dreambooth-Stable-Diffusion/regularization_images/\" + dataset\n",
    "\n",
    "!rm -rf training_images/.ipynb_checkpoints\n",
    "!python \"main.py\" \\\n",
    " --base configs/stable-diffusion/v1-finetune_unfrozen.yaml \\\n",
    " -t \\\n",
    " --actual_resume \"model.ckpt\" \\\n",
    " --reg_data_root \"{reg_data_root}\" \\\n",
    " -n \"{project_name}\" \\\n",
    " --gpus 0, \\\n",
    " --data_root \"/workspace/Dreambooth-Stable-Diffusion/training_images\" \\\n",
    " --max_training_steps {max_training_steps} \\\n",
    " --class_word \"{class_word}\" \\\n",
    " --token \"{token}\" \\\n",
    " --no-test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0807faa-4e6f-49e7-9a3d-27bb3f9cbb3d",
   "metadata": {},
   "outputs": [],
   "source": [
    "token = \"freya556\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc49d0bd",
   "metadata": {},
   "source": [
    "## Copy and name the checkpoint file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "80236388",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "# Copy the checkpoint into our `trained_models` folder\n",
    "\n",
    "directory_paths = !ls -d logs/*\n",
    "last_checkpoint_file = directory_paths[-1] + \"/checkpoints/last.ckpt\"\n",
    "training_images = !find training_images/*\n",
    "date_string = !date +\"%Y-%m-%dT%H-%M-%S\"\n",
    "file_name = date_string[-1] + \"_\" + project_name + \"_\" + str(len(training_images)) + \"_training_images_\" +  str(max_training_steps) + \"_max_training_steps_\" + token + \"_token_\" + class_word + \"_class_word.ckpt\"\n",
    "\n",
    "file_name = file_name.replace(\" \", \"_\")\n",
    "\n",
    "!mkdir -p trained_models\n",
    "!mv \"{last_checkpoint_file}\" \"trained_models/{file_name}\"\n",
    "\n",
    "print(\"Download your trained model file from trained_models/\" + file_name + \" and use in your favorite Stable Diffusion repo!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7acdfe3a-106a-44b3-aaaa-c534c718cf8d",
   "metadata": {},
   "outputs": [],
   "source": [
    "#file_name = \"2022-11-02T16-10-52_project_name_23_training_images_2020_max_training_steps_freya556_token_person_class_word.ckpt\"\n",
    "print(file_name)\n",
    "print(reg_data_root)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9394704",
   "metadata": {},
   "source": [
    "# Optional - Upload to google drive\n",
    "* run the following commands in a new `terminal` in the `Dreambooth-Stable-Diffusion` directory\n",
    "* `chmod +x ./gdrive`\n",
    "* `./gdrive about`\n",
    "* `paste your token here after navigating to the link`\n",
    "* `./gdrive upload trained_models/{file_name.ckpt}`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a90ac5c",
   "metadata": {},
   "source": [
    "# Big Important Note!\n",
    "\n",
    "The way to use your token is `<token> <class>` ie `joepenna person` and not just `joepenna`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d28d0139",
   "metadata": {},
   "source": [
    "## Generate Images With Your Trained Model!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "id": "80ddb03b",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Global seed set to 42\n",
      "Loading model from /workspace/Dreambooth-Stable-Diffusion/trained_models/2022-11-05T20-22-06_project_name_17_training_images_2010_max_training_steps_freya556_token_person_class_word.ckpt\n",
      "Global Step: 2000\n",
      "LatentDiffusion: Running in eps-prediction mode\n",
      "DiffusionWrapper has 859.52 M params.\n",
      "making attention of type 'vanilla' with 512 in_channels\n",
      "Working with z of shape (1, 4, 32, 32) = 4096 dimensions.\n",
      "making attention of type 'vanilla' with 512 in_channels\n",
      "Some weights of the model checkpoint at openai/clip-vit-large-patch14 were not used when initializing CLIPTextModel: ['vision_model.encoder.layers.9.mlp.fc1.bias', 'vision_model.encoder.layers.2.layer_norm1.bias', 'vision_model.encoder.layers.1.self_attn.q_proj.bias', 'vision_model.encoder.layers.20.layer_norm2.weight', 'vision_model.encoder.layers.9.self_attn.q_proj.weight', 'vision_model.encoder.layers.23.mlp.fc2.weight', 'vision_model.encoder.layers.22.self_attn.q_proj.weight', 'vision_model.encoder.layers.0.mlp.fc1.weight', 'vision_model.encoder.layers.9.self_attn.v_proj.bias', 'vision_model.encoder.layers.11.layer_norm1.bias', 'vision_model.encoder.layers.21.self_attn.q_proj.bias', 'vision_model.encoder.layers.16.self_attn.out_proj.weight', 'vision_model.encoder.layers.0.self_attn.v_proj.bias', 'vision_model.encoder.layers.23.mlp.fc1.bias', 'vision_model.encoder.layers.22.layer_norm1.bias', 'vision_model.encoder.layers.16.mlp.fc2.weight', 'vision_model.encoder.layers.1.self_attn.k_proj.bias', 'vision_model.encoder.layers.20.mlp.fc2.bias', 'vision_model.encoder.layers.12.self_attn.v_proj.bias', 'vision_model.encoder.layers.10.self_attn.v_proj.bias', 'vision_model.encoder.layers.16.self_attn.v_proj.bias', 'vision_model.encoder.layers.13.self_attn.out_proj.bias', 'vision_model.encoder.layers.6.self_attn.out_proj.weight', 'vision_model.encoder.layers.8.self_attn.v_proj.weight', 'vision_model.encoder.layers.22.self_attn.out_proj.bias', 'vision_model.encoder.layers.10.mlp.fc2.weight', 'vision_model.encoder.layers.5.layer_norm2.bias', 'vision_model.encoder.layers.17.mlp.fc1.weight', 'vision_model.encoder.layers.11.layer_norm1.weight', 'vision_model.encoder.layers.18.mlp.fc1.bias', 'vision_model.encoder.layers.22.self_attn.out_proj.weight', 'vision_model.encoder.layers.21.self_attn.k_proj.weight', 'vision_model.encoder.layers.5.self_attn.out_proj.bias', 'vision_model.encoder.layers.3.self_attn.v_proj.weight', 'vision_model.encoder.layers.12.layer_norm2.weight', 'vision_model.encoder.layers.14.self_attn.q_proj.bias', 'vision_model.encoder.layers.11.mlp.fc1.bias', 'vision_model.encoder.layers.13.layer_norm2.bias', 'vision_model.encoder.layers.11.mlp.fc1.weight', 'vision_model.encoder.layers.13.mlp.fc1.bias', 'vision_model.encoder.layers.16.mlp.fc1.weight', 'vision_model.encoder.layers.0.layer_norm2.bias', 'vision_model.encoder.layers.9.layer_norm1.bias', 'vision_model.encoder.layers.21.layer_norm1.weight', 'vision_model.encoder.layers.4.self_attn.out_proj.weight', 'vision_model.encoder.layers.22.layer_norm2.weight', 'vision_model.encoder.layers.0.self_attn.out_proj.bias', 'vision_model.encoder.layers.13.mlp.fc2.bias', 'vision_model.encoder.layers.23.self_attn.k_proj.weight', 'vision_model.encoder.layers.18.self_attn.v_proj.bias', 'vision_model.encoder.layers.1.layer_norm1.weight', 'vision_model.encoder.layers.4.layer_norm2.weight', 'vision_model.encoder.layers.14.self_attn.out_proj.bias', 'vision_model.encoder.layers.6.self_attn.q_proj.bias', 'vision_model.encoder.layers.11.self_attn.v_proj.bias', 'vision_model.encoder.layers.18.self_attn.out_proj.weight', 'vision_model.encoder.layers.18.layer_norm1.bias', 'vision_model.encoder.layers.15.mlp.fc1.bias', 'vision_model.encoder.layers.22.mlp.fc2.bias', 'vision_model.encoder.layers.15.self_attn.k_proj.bias', 'vision_model.encoder.layers.2.mlp.fc2.weight', 'vision_model.encoder.layers.0.mlp.fc2.bias', 'vision_model.encoder.layers.7.self_attn.k_proj.bias', 'vision_model.encoder.layers.9.layer_norm2.bias', 'vision_model.encoder.layers.22.mlp.fc2.weight', 'vision_model.encoder.layers.0.layer_norm2.weight', 'vision_model.encoder.layers.15.self_attn.out_proj.weight', 'vision_model.encoder.layers.21.mlp.fc2.bias', 'vision_model.encoder.layers.6.mlp.fc2.weight', 'vision_model.encoder.layers.4.layer_norm1.bias', 'vision_model.encoder.layers.21.mlp.fc1.weight', 'vision_model.encoder.layers.1.layer_norm1.bias', 'vision_model.encoder.layers.17.layer_norm1.bias', 'vision_model.encoder.layers.19.mlp.fc1.weight', 'vision_model.encoder.layers.22.self_attn.k_proj.weight', 'vision_model.encoder.layers.10.layer_norm2.bias', 'vision_model.encoder.layers.15.layer_norm2.weight', 'vision_model.encoder.layers.2.mlp.fc1.bias', 'vision_model.encoder.layers.18.self_attn.k_proj.weight', 'vision_model.encoder.layers.7.self_attn.q_proj.bias', 'vision_model.encoder.layers.6.self_attn.k_proj.weight', 'vision_model.encoder.layers.14.mlp.fc1.bias', 'vision_model.encoder.layers.6.layer_norm2.weight', 'vision_model.encoder.layers.3.mlp.fc1.bias', 'vision_model.encoder.layers.14.layer_norm2.weight', 'vision_model.encoder.layers.13.self_attn.q_proj.weight', 'vision_model.encoder.layers.7.self_attn.out_proj.bias', 'vision_model.encoder.layers.21.layer_norm2.weight', 'vision_model.encoder.layers.18.layer_norm1.weight', 'vision_model.encoder.layers.2.layer_norm2.weight', 'vision_model.encoder.layers.9.mlp.fc2.bias', 'vision_model.encoder.layers.10.self_attn.out_proj.weight', 'vision_model.encoder.layers.3.self_attn.v_proj.bias', 'vision_model.encoder.layers.12.self_attn.q_proj.weight', 'vision_model.encoder.layers.20.self_attn.v_proj.weight', 'vision_model.encoder.layers.4.self_attn.k_proj.weight', 'vision_model.encoder.layers.21.layer_norm1.bias', 'vision_model.encoder.layers.4.mlp.fc2.weight', 'vision_model.encoder.layers.13.mlp.fc1.weight', 'vision_model.encoder.layers.5.self_attn.q_proj.bias', 'vision_model.encoder.layers.15.mlp.fc2.weight', 'vision_model.encoder.layers.22.self_attn.v_proj.weight', 'vision_model.encoder.layers.6.self_attn.v_proj.weight', 'vision_model.encoder.layers.21.self_attn.v_proj.bias', 'vision_model.encoder.layers.4.mlp.fc1.bias', 'vision_model.encoder.layers.1.mlp.fc1.bias', 'vision_model.encoder.layers.5.mlp.fc1.bias', 'vision_model.encoder.layers.15.mlp.fc1.weight', 'vision_model.encoder.layers.3.layer_norm1.bias', 'vision_model.encoder.layers.23.self_attn.q_proj.bias', 'vision_model.encoder.layers.18.mlp.fc2.bias', 'vision_model.encoder.layers.23.self_attn.k_proj.bias', 'vision_model.encoder.layers.17.self_attn.q_proj.bias', 'vision_model.encoder.layers.15.self_attn.out_proj.bias', 'vision_model.encoder.layers.6.self_attn.out_proj.bias', 'vision_model.encoder.layers.18.self_attn.q_proj.weight', 'vision_model.encoder.layers.12.layer_norm2.bias', 'vision_model.encoder.layers.8.self_attn.k_proj.bias', 'vision_model.encoder.layers.9.self_attn.v_proj.weight', 'vision_model.encoder.layers.12.mlp.fc1.weight', 'vision_model.encoder.layers.14.layer_norm2.bias', 'vision_model.encoder.layers.2.layer_norm2.bias', 'vision_model.encoder.layers.10.mlp.fc1.weight', 'vision_model.encoder.layers.19.self_attn.q_proj.weight', 'vision_model.encoder.layers.11.self_attn.out_proj.weight', 'vision_model.encoder.layers.19.self_attn.k_proj.bias', 'vision_model.encoder.layers.5.self_attn.k_proj.weight', 'vision_model.encoder.layers.20.self_attn.q_proj.bias', 'vision_model.encoder.layers.16.self_attn.v_proj.weight', 'vision_model.encoder.layers.2.self_attn.k_proj.weight', 'vision_model.encoder.layers.8.mlp.fc1.bias', 'vision_model.encoder.layers.20.mlp.fc2.weight', 'vision_model.pre_layrnorm.weight', 'vision_model.encoder.layers.21.self_attn.out_proj.weight', 'vision_model.encoder.layers.18.layer_norm2.weight', 'vision_model.encoder.layers.9.self_attn.out_proj.bias', 'vision_model.encoder.layers.10.layer_norm1.bias', 'vision_model.encoder.layers.4.self_attn.out_proj.bias', 'vision_model.encoder.layers.11.layer_norm2.weight', 'vision_model.encoder.layers.7.self_attn.v_proj.weight', 'vision_model.encoder.layers.6.mlp.fc2.bias', 'vision_model.encoder.layers.3.mlp.fc1.weight', 'vision_model.encoder.layers.8.layer_norm1.bias', 'vision_model.encoder.layers.23.mlp.fc1.weight', 'vision_model.encoder.layers.2.self_attn.out_proj.bias', 'vision_model.encoder.layers.2.mlp.fc1.weight', 'vision_model.encoder.layers.14.mlp.fc2.bias', 'vision_model.encoder.layers.6.layer_norm1.bias', 'vision_model.encoder.layers.6.self_attn.q_proj.weight', 'vision_model.encoder.layers.21.self_attn.v_proj.weight', 'vision_model.encoder.layers.5.self_attn.k_proj.bias', 'vision_model.encoder.layers.8.layer_norm1.weight', 'vision_model.encoder.layers.17.mlp.fc1.bias', 'vision_model.embeddings.class_embedding', 'vision_model.encoder.layers.8.layer_norm2.bias', 'vision_model.encoder.layers.6.layer_norm2.bias', 'vision_model.encoder.layers.7.self_attn.q_proj.weight', 'vision_model.encoder.layers.20.self_attn.v_proj.bias', 'vision_model.encoder.layers.19.self_attn.out_proj.weight', 'vision_model.encoder.layers.23.self_attn.out_proj.bias', 'vision_model.encoder.layers.23.self_attn.out_proj.weight', 'vision_model.encoder.layers.1.layer_norm2.weight', 'logit_scale', 'vision_model.encoder.layers.10.mlp.fc1.bias', 'vision_model.encoder.layers.17.self_attn.k_proj.bias', 'vision_model.encoder.layers.5.layer_norm1.weight', 'vision_model.encoder.layers.12.self_attn.k_proj.bias', 'vision_model.encoder.layers.4.self_attn.q_proj.weight', 'vision_model.encoder.layers.7.self_attn.v_proj.bias', 'vision_model.encoder.layers.0.layer_norm1.weight', 'vision_model.encoder.layers.20.mlp.fc1.weight', 'vision_model.encoder.layers.16.self_attn.q_proj.weight', 'vision_model.encoder.layers.20.self_attn.k_proj.weight', 'vision_model.encoder.layers.7.layer_norm2.weight', 'vision_model.encoder.layers.13.mlp.fc2.weight', 'vision_model.encoder.layers.5.layer_norm2.weight', 'vision_model.encoder.layers.19.layer_norm2.weight', 'vision_model.encoder.layers.6.layer_norm1.weight', 'vision_model.encoder.layers.16.self_attn.q_proj.bias', 'vision_model.encoder.layers.18.mlp.fc1.weight', 'vision_model.encoder.layers.20.layer_norm1.bias', 'vision_model.encoder.layers.2.layer_norm1.weight', 'vision_model.encoder.layers.13.self_attn.v_proj.bias', 'vision_model.encoder.layers.11.mlp.fc2.weight', 'vision_model.encoder.layers.12.self_attn.k_proj.weight', 'vision_model.encoder.layers.17.mlp.fc2.bias', 'vision_model.encoder.layers.3.self_attn.k_proj.weight', 'vision_model.encoder.layers.13.layer_norm2.weight', 'vision_model.encoder.layers.14.self_attn.q_proj.weight', 'vision_model.encoder.layers.3.mlp.fc2.bias', 'vision_model.encoder.layers.6.mlp.fc1.weight', 'vision_model.encoder.layers.14.self_attn.v_proj.weight', 'vision_model.encoder.layers.17.self_attn.out_proj.weight', 'vision_model.encoder.layers.17.self_attn.out_proj.bias', 'vision_model.encoder.layers.15.layer_norm1.bias', 'vision_model.encoder.layers.13.self_attn.k_proj.bias', 'vision_model.encoder.layers.3.layer_norm2.bias', 'vision_model.encoder.layers.16.self_attn.out_proj.bias', 'vision_model.encoder.layers.19.self_attn.q_proj.bias', 'vision_model.encoder.layers.0.self_attn.k_proj.weight', 'vision_model.encoder.layers.22.mlp.fc1.bias', 'vision_model.encoder.layers.22.layer_norm2.bias', 'vision_model.encoder.layers.20.layer_norm1.weight', 'vision_model.encoder.layers.13.self_attn.k_proj.weight', 'vision_model.encoder.layers.17.mlp.fc2.weight', 'vision_model.encoder.layers.2.self_attn.q_proj.bias', 'vision_model.encoder.layers.15.self_attn.v_proj.bias', 'vision_model.encoder.layers.4.mlp.fc1.weight', 'vision_model.encoder.layers.13.self_attn.v_proj.weight', 'vision_model.encoder.layers.7.layer_norm1.bias', 'vision_model.encoder.layers.21.self_attn.k_proj.bias', 'vision_model.encoder.layers.5.mlp.fc1.weight', 'vision_model.encoder.layers.9.mlp.fc2.weight', 'vision_model.encoder.layers.11.self_attn.k_proj.weight', 'vision_model.encoder.layers.4.layer_norm2.bias', 'vision_model.encoder.layers.17.layer_norm1.weight', 'vision_model.encoder.layers.21.self_attn.out_proj.bias', 'vision_model.encoder.layers.19.self_attn.k_proj.weight', 'vision_model.encoder.layers.1.mlp.fc2.weight', 'vision_model.encoder.layers.7.self_attn.out_proj.weight', 'vision_model.encoder.layers.22.layer_norm1.weight', 'vision_model.encoder.layers.12.mlp.fc1.bias', 'vision_model.encoder.layers.3.self_attn.out_proj.weight', 'vision_model.encoder.layers.5.layer_norm1.bias', 'vision_model.embeddings.position_embedding.weight', 'vision_model.post_layernorm.weight', 'vision_model.encoder.layers.18.mlp.fc2.weight', 'vision_model.encoder.layers.18.self_attn.v_proj.weight', 'vision_model.encoder.layers.8.mlp.fc2.bias', 'vision_model.encoder.layers.6.mlp.fc1.bias', 'vision_model.encoder.layers.8.mlp.fc1.weight', 'vision_model.encoder.layers.5.self_attn.v_proj.weight', 'vision_model.encoder.layers.14.self_attn.out_proj.weight', 'vision_model.encoder.layers.19.self_attn.v_proj.bias', 'vision_model.encoder.layers.10.self_attn.out_proj.bias', 'vision_model.encoder.layers.1.self_attn.v_proj.bias', 'vision_model.encoder.layers.4.layer_norm1.weight', 'vision_model.encoder.layers.5.mlp.fc2.bias', 'vision_model.encoder.layers.7.mlp.fc1.bias', 'vision_model.encoder.layers.17.self_attn.v_proj.weight', 'vision_model.encoder.layers.12.mlp.fc2.bias', 'vision_model.encoder.layers.2.self_attn.q_proj.weight', 'vision_model.encoder.layers.22.self_attn.k_proj.bias', 'vision_model.encoder.layers.20.mlp.fc1.bias', 'vision_model.encoder.layers.17.layer_norm2.bias', 'vision_model.encoder.layers.18.self_attn.k_proj.bias', 'vision_model.encoder.layers.15.self_attn.q_proj.weight', 'vision_model.encoder.layers.7.self_attn.k_proj.weight', 'vision_model.encoder.layers.1.self_attn.k_proj.weight', 'vision_model.encoder.layers.13.self_attn.out_proj.weight', 'vision_model.encoder.layers.16.self_attn.k_proj.weight', 'vision_model.encoder.layers.10.self_attn.v_proj.weight', 'vision_model.encoder.layers.10.mlp.fc2.bias', 'vision_model.encoder.layers.22.self_attn.q_proj.bias', 'vision_model.encoder.layers.13.layer_norm1.weight', 'vision_model.encoder.layers.15.self_attn.q_proj.bias', 'vision_model.post_layernorm.bias', 'vision_model.encoder.layers.19.mlp.fc2.bias', 'vision_model.encoder.layers.0.self_attn.v_proj.weight', 'vision_model.encoder.layers.22.mlp.fc1.weight', 'vision_model.encoder.layers.12.layer_norm1.bias', 'vision_model.encoder.layers.21.mlp.fc2.weight', 'vision_model.encoder.layers.8.self_attn.q_proj.bias', 'vision_model.encoder.layers.15.self_attn.v_proj.weight', 'vision_model.encoder.layers.3.self_attn.q_proj.weight', 'vision_model.embeddings.position_ids', 'vision_model.encoder.layers.12.layer_norm1.weight', 'vision_model.encoder.layers.15.mlp.fc2.bias', 'vision_model.encoder.layers.11.self_attn.v_proj.weight', 'vision_model.encoder.layers.19.mlp.fc1.bias', 'vision_model.encoder.layers.11.layer_norm2.bias', 'vision_model.encoder.layers.21.mlp.fc1.bias', 'vision_model.encoder.layers.0.self_attn.out_proj.weight', 'vision_model.encoder.layers.15.self_attn.k_proj.weight', 'vision_model.encoder.layers.12.self_attn.v_proj.weight', 'vision_model.encoder.layers.20.layer_norm2.bias', 'vision_model.encoder.layers.23.mlp.fc2.bias', 'vision_model.encoder.layers.1.mlp.fc2.bias', 'vision_model.encoder.layers.17.self_attn.q_proj.weight', 'vision_model.encoder.layers.14.layer_norm1.weight', 'vision_model.encoder.layers.22.self_attn.v_proj.bias', 'vision_model.encoder.layers.3.self_attn.k_proj.bias', 'vision_model.encoder.layers.0.layer_norm1.bias', 'vision_model.encoder.layers.9.self_attn.q_proj.bias', 'vision_model.encoder.layers.16.layer_norm2.weight', 'vision_model.encoder.layers.6.self_attn.k_proj.bias', 'vision_model.encoder.layers.1.layer_norm2.bias', 'vision_model.encoder.layers.16.mlp.fc1.bias', 'vision_model.encoder.layers.7.layer_norm2.bias', 'vision_model.encoder.layers.8.self_attn.out_proj.weight', 'vision_model.encoder.layers.9.layer_norm1.weight', 'vision_model.encoder.layers.4.self_attn.v_proj.weight', 'vision_model.encoder.layers.7.mlp.fc2.bias', 'vision_model.encoder.layers.10.self_attn.q_proj.weight', 'vision_model.encoder.layers.10.self_attn.q_proj.bias', 'vision_model.encoder.layers.8.self_attn.v_proj.bias', 'vision_model.encoder.layers.11.self_attn.q_proj.bias', 'text_projection.weight', 'vision_model.encoder.layers.17.self_attn.v_proj.bias', 'vision_model.encoder.layers.8.self_attn.k_proj.weight', 'vision_model.encoder.layers.23.layer_norm2.weight', 'vision_model.encoder.layers.8.mlp.fc2.weight', 'vision_model.encoder.layers.16.layer_norm2.bias', 'vision_model.encoder.layers.12.self_attn.q_proj.bias', 'vision_model.encoder.layers.1.mlp.fc1.weight', 'vision_model.encoder.layers.2.self_attn.v_proj.bias', 'vision_model.encoder.layers.14.self_attn.k_proj.weight', 'vision_model.encoder.layers.14.mlp.fc1.weight', 'vision_model.encoder.layers.7.mlp.fc2.weight', 'vision_model.encoder.layers.3.layer_norm2.weight', 'vision_model.encoder.layers.4.self_attn.q_proj.bias', 'vision_model.encoder.layers.2.self_attn.v_proj.weight', 'vision_model.encoder.layers.3.self_attn.out_proj.bias', 'vision_model.encoder.layers.23.layer_norm2.bias', 'vision_model.encoder.layers.3.mlp.fc2.weight', 'vision_model.encoder.layers.3.layer_norm1.weight', 'vision_model.encoder.layers.8.layer_norm2.weight', 'vision_model.encoder.layers.16.layer_norm1.weight', 'vision_model.encoder.layers.5.self_attn.v_proj.bias', 'vision_model.encoder.layers.23.layer_norm1.weight', 'vision_model.encoder.layers.18.layer_norm2.bias', 'vision_model.encoder.layers.14.self_attn.v_proj.bias', 'vision_model.encoder.layers.9.self_attn.k_proj.bias', 'vision_model.encoder.layers.19.layer_norm2.bias', 'vision_model.encoder.layers.9.self_attn.k_proj.weight', 'vision_model.encoder.layers.10.self_attn.k_proj.weight', 'vision_model.encoder.layers.18.self_attn.q_proj.bias', 'vision_model.encoder.layers.19.self_attn.v_proj.weight', 'vision_model.encoder.layers.14.layer_norm1.bias', 'vision_model.encoder.layers.1.self_attn.out_proj.weight', 'vision_model.encoder.layers.23.self_attn.v_proj.bias', 'vision_model.encoder.layers.11.mlp.fc2.bias', 'vision_model.encoder.layers.19.mlp.fc2.weight', 'vision_model.encoder.layers.2.self_attn.k_proj.bias', 'vision_model.encoder.layers.10.self_attn.k_proj.bias', 'vision_model.encoder.layers.7.layer_norm1.weight', 'vision_model.encoder.layers.16.mlp.fc2.bias', 'vision_model.encoder.layers.11.self_attn.out_proj.bias', 'vision_model.encoder.layers.3.self_attn.q_proj.bias', 'vision_model.encoder.layers.15.layer_norm2.bias', 'vision_model.encoder.layers.11.self_attn.q_proj.weight', 'vision_model.encoder.layers.21.self_attn.q_proj.weight', 'vision_model.encoder.layers.17.layer_norm2.weight', 'vision_model.encoder.layers.0.mlp.fc1.bias', 'vision_model.encoder.layers.19.self_attn.out_proj.bias', 'vision_model.encoder.layers.14.self_attn.k_proj.bias', 'vision_model.encoder.layers.12.self_attn.out_proj.weight', 'vision_model.encoder.layers.0.self_attn.q_proj.bias', 'vision_model.encoder.layers.10.layer_norm2.weight', 'vision_model.encoder.layers.13.self_attn.q_proj.bias', 'vision_model.encoder.layers.20.self_attn.k_proj.bias', 'vision_model.encoder.layers.20.self_attn.out_proj.bias', 'vision_model.encoder.layers.18.self_attn.out_proj.bias', 'vision_model.encoder.layers.1.self_attn.v_proj.weight', 'vision_model.encoder.layers.9.self_attn.out_proj.weight', 'vision_model.encoder.layers.20.self_attn.out_proj.weight', 'vision_model.encoder.layers.4.mlp.fc2.bias', 'vision_model.encoder.layers.0.self_attn.k_proj.bias', 'vision_model.encoder.layers.5.self_attn.q_proj.weight', 'vision_model.encoder.layers.9.layer_norm2.weight', 'vision_model.encoder.layers.1.self_attn.out_proj.bias', 'vision_model.encoder.layers.1.self_attn.q_proj.weight', 'visual_projection.weight', 'vision_model.encoder.layers.8.self_attn.q_proj.weight', 'vision_model.encoder.layers.11.self_attn.k_proj.bias', 'vision_model.encoder.layers.4.self_attn.k_proj.bias', 'vision_model.encoder.layers.2.mlp.fc2.bias', 'vision_model.encoder.layers.12.self_attn.out_proj.bias', 'vision_model.encoder.layers.5.mlp.fc2.weight', 'vision_model.encoder.layers.8.self_attn.out_proj.bias', 'vision_model.encoder.layers.10.layer_norm1.weight', 'vision_model.encoder.layers.14.mlp.fc2.weight', 'vision_model.encoder.layers.5.self_attn.out_proj.weight', 'vision_model.encoder.layers.15.layer_norm1.weight', 'vision_model.encoder.layers.20.self_attn.q_proj.weight', 'vision_model.encoder.layers.19.layer_norm1.weight', 'vision_model.encoder.layers.16.self_attn.k_proj.bias', 'vision_model.encoder.layers.4.self_attn.v_proj.bias', 'vision_model.encoder.layers.12.mlp.fc2.weight', 'vision_model.encoder.layers.13.layer_norm1.bias', 'vision_model.encoder.layers.17.self_attn.k_proj.weight', 'vision_model.encoder.layers.9.mlp.fc1.weight', 'vision_model.encoder.layers.23.layer_norm1.bias', 'vision_model.embeddings.patch_embedding.weight', 'vision_model.encoder.layers.16.layer_norm1.bias', 'vision_model.encoder.layers.19.layer_norm1.bias', 'vision_model.encoder.layers.0.mlp.fc2.weight', 'vision_model.encoder.layers.2.self_attn.out_proj.weight', 'vision_model.encoder.layers.21.layer_norm2.bias', 'vision_model.pre_layrnorm.bias', 'vision_model.encoder.layers.0.self_attn.q_proj.weight', 'vision_model.encoder.layers.23.self_attn.q_proj.weight', 'vision_model.encoder.layers.7.mlp.fc1.weight', 'vision_model.encoder.layers.23.self_attn.v_proj.weight', 'vision_model.encoder.layers.6.self_attn.v_proj.bias']\n",
      "- This IS expected if you are initializing CLIPTextModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
      "- This IS NOT expected if you are initializing CLIPTextModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
      "Sampling:   0%|                                           | 0/2 [00:00<?, ?it/s]\n",
      "data:   0%|                                               | 0/1 [00:00<?, ?it/s]\u001b[AData shape for DDIM sampling is (8, 4, 64, 64), eta 0.0\n",
      "Running DDIM Sampling with 40 timesteps\n",
      "\n",
      "\n",
      "DDIM Sampler:   0%|                                      | 0/40 [00:00<?, ?it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:   2%|▊                             | 1/40 [00:01<01:09,  1.78s/it]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:   5%|█▌                            | 2/40 [00:02<00:41,  1.09s/it]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:   8%|██▎                           | 3/40 [00:03<00:32,  1.14it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  10%|███                           | 4/40 [00:03<00:27,  1.30it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  12%|███▊                          | 5/40 [00:04<00:24,  1.40it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  15%|████▌                         | 6/40 [00:04<00:23,  1.47it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  18%|█████▎                        | 7/40 [00:05<00:21,  1.52it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  20%|██████                        | 8/40 [00:06<00:20,  1.55it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  22%|██████▊                       | 9/40 [00:06<00:19,  1.58it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  25%|███████▎                     | 10/40 [00:07<00:18,  1.59it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  28%|███████▉                     | 11/40 [00:07<00:18,  1.61it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  30%|████████▋                    | 12/40 [00:08<00:17,  1.61it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  32%|█████████▍                   | 13/40 [00:09<00:16,  1.62it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  35%|██████████▏                  | 14/40 [00:09<00:16,  1.62it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  38%|██████████▉                  | 15/40 [00:10<00:15,  1.62it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  40%|███████████▌                 | 16/40 [00:10<00:14,  1.62it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  42%|████████████▎                | 17/40 [00:11<00:14,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  45%|█████████████                | 18/40 [00:12<00:13,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  48%|█████████████▊               | 19/40 [00:12<00:12,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  50%|██████████████▌              | 20/40 [00:13<00:12,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  52%|███████████████▏             | 21/40 [00:14<00:11,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  55%|███████████████▉             | 22/40 [00:14<00:11,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  57%|████████████████▋            | 23/40 [00:15<00:10,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  60%|█████████████████▍           | 24/40 [00:15<00:09,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  62%|██████████████████▏          | 25/40 [00:16<00:09,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  65%|██████████████████▊          | 26/40 [00:17<00:08,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  68%|███████████████████▌         | 27/40 [00:17<00:07,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  70%|████████████████████▎        | 28/40 [00:18<00:07,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  72%|█████████████████████        | 29/40 [00:18<00:06,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  75%|█████████████████████▊       | 30/40 [00:19<00:06,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  78%|██████████████████████▍      | 31/40 [00:20<00:05,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  80%|███████████████████████▏     | 32/40 [00:20<00:04,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  82%|███████████████████████▉     | 33/40 [00:21<00:04,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  85%|████████████████████████▋    | 34/40 [00:22<00:03,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  88%|█████████████████████████▍   | 35/40 [00:22<00:03,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  90%|██████████████████████████   | 36/40 [00:23<00:02,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  92%|██████████████████████████▊  | 37/40 [00:23<00:01,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  95%|███████████████████████████▌ | 38/40 [00:24<00:01,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  98%|████████████████████████████▎| 39/40 [00:25<00:00,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler: 100%|█████████████████████████████| 40/40 [00:25<00:00,  1.56it/s]\u001b[A\u001b[A\n",
      "\n",
      "data: 100%|███████████████████████████████████████| 1/1 [00:27<00:00, 27.20s/it]\u001b[A\n",
      "Sampling:  50%|█████████████████▌                 | 1/2 [00:27<00:27, 27.20s/it]\n",
      "data:   0%|                                               | 0/1 [00:00<?, ?it/s]\u001b[AData shape for DDIM sampling is (8, 4, 64, 64), eta 0.0\n",
      "Running DDIM Sampling with 40 timesteps\n",
      "\n",
      "\n",
      "DDIM Sampler:   0%|                                      | 0/40 [00:00<?, ?it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:   2%|▊                             | 1/40 [00:00<00:23,  1.65it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:   5%|█▌                            | 2/40 [00:01<00:23,  1.64it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:   8%|██▎                           | 3/40 [00:01<00:22,  1.64it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  10%|███                           | 4/40 [00:02<00:21,  1.64it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  12%|███▊                          | 5/40 [00:03<00:21,  1.64it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  15%|████▌                         | 6/40 [00:03<00:20,  1.64it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  18%|█████▎                        | 7/40 [00:04<00:20,  1.64it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  20%|██████                        | 8/40 [00:04<00:19,  1.64it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  22%|██████▊                       | 9/40 [00:05<00:18,  1.64it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  25%|███████▎                     | 10/40 [00:06<00:18,  1.64it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  28%|███████▉                     | 11/40 [00:06<00:17,  1.64it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  30%|████████▋                    | 12/40 [00:07<00:17,  1.64it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  32%|█████████▍                   | 13/40 [00:07<00:16,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  35%|██████████▏                  | 14/40 [00:08<00:15,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  38%|██████████▉                  | 15/40 [00:09<00:15,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  40%|███████████▌                 | 16/40 [00:09<00:14,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  42%|████████████▎                | 17/40 [00:10<00:14,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  45%|█████████████                | 18/40 [00:11<00:13,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  48%|█████████████▊               | 19/40 [00:11<00:12,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  50%|██████████████▌              | 20/40 [00:12<00:12,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  52%|███████████████▏             | 21/40 [00:12<00:11,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  55%|███████████████▉             | 22/40 [00:13<00:11,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  57%|████████████████▋            | 23/40 [00:14<00:10,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  60%|█████████████████▍           | 24/40 [00:14<00:09,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  62%|██████████████████▏          | 25/40 [00:15<00:09,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  65%|██████████████████▊          | 26/40 [00:15<00:08,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  68%|███████████████████▌         | 27/40 [00:16<00:07,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  70%|████████████████████▎        | 28/40 [00:17<00:07,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  72%|█████████████████████        | 29/40 [00:17<00:06,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  75%|█████████████████████▊       | 30/40 [00:18<00:06,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  78%|██████████████████████▍      | 31/40 [00:18<00:05,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  80%|███████████████████████▏     | 32/40 [00:19<00:04,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  82%|███████████████████████▉     | 33/40 [00:20<00:04,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  85%|████████████████████████▋    | 34/40 [00:20<00:03,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  88%|█████████████████████████▍   | 35/40 [00:21<00:03,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  90%|██████████████████████████   | 36/40 [00:22<00:02,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  92%|██████████████████████████▊  | 37/40 [00:22<00:01,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  95%|███████████████████████████▌ | 38/40 [00:23<00:01,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler:  98%|████████████████████████████▎| 39/40 [00:23<00:00,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "DDIM Sampler: 100%|█████████████████████████████| 40/40 [00:24<00:00,  1.63it/s]\u001b[A\u001b[A\n",
      "\n",
      "data: 100%|███████████████████████████████████████| 1/1 [00:25<00:00, 25.27s/it]\u001b[A\n",
      "Sampling: 100%|███████████████████████████████████| 2/2 [00:52<00:00, 26.23s/it]\n",
      "Your samples are ready and waiting for you here: \n",
      "outputs/txt2img-samples \n",
      " \n",
      "Enjoy.\n"
     ]
    }
   ],
   "source": [
    "!python scripts/stable_txt2img.py \\\n",
    " --ddim_eta 0.0 \\\n",
    " --n_samples 8 \\\n",
    " --n_iter 2 \\\n",
    " --scale 7.0 \\\n",
    " --ddim_steps 40 \\\n",
    " --ckpt \"/workspace/Dreambooth-Stable-Diffusion/trained_models/{file_name}\" \\\n",
    " --prompt \"freya556 person in a comic book as the joker, evil looking, 4K, comic book style, marvel universe\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5f57ad21-6bc9-4501-b1ec-ff8a1f79f2b2",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  },
  "vscode": {
   "interpreter": {
    "hash": "b0fa6594d8f4cbf19f97940f81e996739fb7646882a419484c72d19e05852a7e"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}