vision/getting-started/imagen3_image_generation.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ijGzTHJJUCPY"
      },
      "outputs": [],
      "source": [
        "# Copyright 2024 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VEqbX8OhE8y9"
      },
      "source": [
        "# Imagen 3 Image Generation\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/imagen3_image_generation.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Google Colaboratory logo\"><br> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fvision%2Fgetting-started%2Fimagen3_image_generation.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://cloud.google.com/ml-engine/images/colab-enterprise-logo-32px.png\" alt=\"Google Cloud Colab Enterprise logo\"><br> Run in Colab Enterprise\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/vision/getting-started/imagen3_image_generation.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>    \n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/imagen3_image_generation.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\"><br> View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "</table>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G1KDmM_PBAXz"
      },
      "source": [
        "| | |\n",
        "|-|-|\n",
        "|Author(s) | [Katie Nguyen](https://github.com/katiemn) |"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CkHPv2myT2cx"
      },
      "source": [
        "## Overview\n",
        "\n",
        "### Imagen 3\n",
        "\n",
        "Imagen 3 on Vertex AI brings Google's state of the art generative AI capabilities to application developers. Imagen 3 is Google's highest quality text-to-image model to date. It's capable of creating images with astonishing detail. Thus, developers have more control when building next-generation AI products that transform their imagination into high quality visual assets. Learn more about [Imagen on Vertex AI](https://cloud.google.com/vertex-ai/generative-ai/docs/image/overview).\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DrkcqHrrwMAo"
      },
      "source": [
        "In this tutorial, you will learn how to use the Vertex AI SDK for Python to interact with the Imagen 3 and Imagen 3 Fast models to generate images showcasing:\n",
        "\n",
        "- Photorealistic scenes\n",
        "- Text rendered within images\n",
        "- Quality and latency comparisons within the two models"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r11Gu7qNgx1p"
      },
      "source": [
        "## Get started\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "No17Cw5hgx12"
      },
      "source": [
        "### Install Vertex AI SDK for Python\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tFy3H3aPgx12"
      },
      "outputs": [],
      "source": [
        "%pip install --upgrade --user google-cloud-aiplatform"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R5Xep4W9lq-Z"
      },
      "source": [
        "### Restart runtime\n",
        "\n",
        "To use the newly installed packages in this Jupyter runtime, you must restart the runtime. You can do this by running the cell below, which restarts the current kernel.\n",
        "\n",
        "The restart might take a minute or longer. After it's restarted, continue to the next step."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XRvKdaPDTznN"
      },
      "outputs": [],
      "source": [
        "import IPython\n",
        "\n",
        "app = IPython.Application.instance()\n",
        "app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SbmM4z7FOBpM"
      },
      "source": [
        "<div class=\"alert alert-block alert-warning\">\n",
        "<b>⚠️ The kernel is going to restart. Please wait until it is finished before continuing to the next step. ⚠️</b>\n",
        "</div>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dmWOrTJ3gx13"
      },
      "source": [
        "### Authenticate your notebook environment (Colab only)\n",
        "\n",
        "If you are running this notebook on Google Colab, run the following cell to authenticate your environment.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NyKGtVQjgx13"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DF4l8DTdWgPY"
      },
      "source": [
        "### Set Google Cloud project information and initialize Vertex AI SDK\n",
        "\n",
        "To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
        "\n",
        "Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Nqwi-5ufWp_B"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
        "LOCATION = \"us-central1\"  # @param {type:\"string\"}\n",
        "\n",
        "import vertexai\n",
        "\n",
        "vertexai.init(project=PROJECT_ID, location=LOCATION)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ua6PDqB1iBSb"
      },
      "source": [
        "### Import libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yTiDo0lRh6sc"
      },
      "outputs": [],
      "source": [
        "from vertexai.preview.vision_models import ImageGenerationModel"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Sr2Y3lFwKW1M"
      },
      "source": [
        "### Define a helper function"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "r_38e5rRKB6s"
      },
      "outputs": [],
      "source": [
        "import typing\n",
        "\n",
        "import IPython.display\n",
        "from PIL import Image as PIL_Image\n",
        "from PIL import ImageOps as PIL_ImageOps\n",
        "\n",
        "\n",
        "def display_image(\n",
        "    image,\n",
        "    max_width: int = 600,\n",
        "    max_height: int = 350,\n",
        ") -> None:\n",
        "    pil_image = typing.cast(PIL_Image.Image, image._pil_image)\n",
        "    if pil_image.mode != \"RGB\":\n",
        "        # RGB is supported by all Jupyter environments (e.g. RGBA is not yet)\n",
        "        pil_image = pil_image.convert(\"RGB\")\n",
        "    image_width, image_height = pil_image.size\n",
        "    if max_width < image_width or max_height < image_height:\n",
        "        # Resize to display a smaller notebook image\n",
        "        pil_image = PIL_ImageOps.contain(pil_image, (max_width, max_height))\n",
        "    IPython.display.display(pil_image)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VLmwIj2RD0Fx"
      },
      "source": [
        "### Load the image generation models\n",
        "\n",
        "Imagen 3: `imagen-3.0-generate-001`\n",
        "\n",
        "Imagen 3 Fast: `imagen-3.0-fast-generate-001`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "F-gd2ypQhh7K"
      },
      "outputs": [],
      "source": [
        "generation_model = ImageGenerationModel.from_pretrained(\"imagen-3.0-generate-001\")\n",
        "generation_model_fast = ImageGenerationModel.from_pretrained(\n",
        "    \"imagen-3.0-fast-generate-001\"\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mvH2OTrK8IRS"
      },
      "source": [
        "### Imagen 3 & Imagen 3 Fast\n",
        "\n",
        "With Imagen 3, you also have the option to use Imagen 3 Fast. These two model options give you the choice to optimize for quality and latency, depending on your use case.\n",
        "\n",
        "**Imagen 3:** Generates high quality images with natural lighting and increased photorealism.\n",
        "\n",
        "**Imagen 3 Fast:** Suitable for creating brighter images with a higher contrast. Overall, you can see a 40% decrease in latency in Imagen 3 Fast compared to Imagen 2.\n",
        "\n",
        "With Imagen 3 and Imagen 3 Fast, you can also configure the `aspect ratio` to any of the following:\n",
        "* 1:1\n",
        "* 9:16\n",
        "* 16:9\n",
        "* 3:4\n",
        "* 4:3"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KJqHuGqi31rE"
      },
      "outputs": [],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "prompt = \"\"\"\n",
        "a photorealistic image of the inside of an amethyst crystal on display in a museum\n",
        "\"\"\"\n",
        "\n",
        "# Imagen 3 image generation\n",
        "image = generation_model.generate_images(\n",
        "    prompt=prompt,\n",
        "    number_of_images=1,\n",
        "    aspect_ratio=\"3:4\",\n",
        "    safety_filter_level=\"block_some\",\n",
        "    person_generation=\"allow_adult\",\n",
        ")\n",
        "\n",
        "# Imagen 3 Fast image generation\n",
        "fast_image = generation_model_fast.generate_images(\n",
        "    prompt=prompt,\n",
        "    number_of_images=1,\n",
        "    aspect_ratio=\"3:4\",\n",
        "    safety_filter_level=\"block_some\",\n",
        "    person_generation=\"allow_adult\",\n",
        ")\n",
        "\n",
        "# Display generated images\n",
        "fig, axis = plt.subplots(1, 2, figsize=(12, 6))\n",
        "axis[0].imshow(image[0]._pil_image)\n",
        "axis[0].set_title(\"Imagen 3\")\n",
        "axis[1].imshow(fast_image[0]._pil_image)\n",
        "axis[1].set_title(\"Imagen 3 Fast\")\n",
        "for ax in axis:\n",
        "    ax.axis(\"off\")\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UND0wrdf3mGS"
      },
      "source": [
        "### Photorealism and prompt understanding\n",
        "\n",
        "**Photorealism:** Imagen 3 is capable of generating photorealistic, lifelike images with fewer distracting visual artifacts than our previous models. This increased quality is especially prevalent when generating images of multiple people, animals, and landscapes.\n",
        "\n",
        "**Prompt adherence:** It's also better at understanding natural language and the intent behind your prompts. Thus, they can be written in everyday language and can include specific details including camera angles, lens types, lighting, and stylistic features.\n",
        "\n",
        "When generating images of people you can also set the `safety_filter_level` and `person_generation` parameters accordingly:\n",
        "* `person_generation`: Allow (All ages), Allow (Adults only), Don't allow\n",
        "* `safety_filter_level`: Block most, Block some, Block few"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7JBtrCfd1idI"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "a photorealistic image of a happy family sitting around a table eating dinner, a yellow dog is at their feet begging for food,\n",
        "the table is brown with white comfy chairs,\n",
        "a grand chandelier above their heads casting a warm glow on the family,\n",
        "they are eating pizza and salad,\n",
        "large windows behind them with green curtains pulled off to either side\n",
        "\"\"\"\n",
        "\n",
        "images = generation_model.generate_images(\n",
        "    prompt=prompt,\n",
        "    number_of_images=1,\n",
        "    aspect_ratio=\"1:1\",\n",
        "    safety_filter_level=\"block_some\",\n",
        "    person_generation=\"allow_adult\",\n",
        ")\n",
        "\n",
        "display_image(images[0])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hAeNPdyU9AlC"
      },
      "source": [
        "### Better text rendering\n",
        "\n",
        "Imagen 3 also does a great job accurately rendering small words and phrases in images. This could be particularly useful for generating business cards, posters, banners, product designs, or greeting cards."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DLmdtT8Q38fn"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "a beige baseball cap with 'good vibes' written on top in white bubbly stitched letters that are outlined in neon green,\n",
        "display it against a pool background with palm trees and pool floats\n",
        "\"\"\"\n",
        "\n",
        "images = generation_model.generate_images(\n",
        "    prompt=prompt,\n",
        "    number_of_images=1,\n",
        "    aspect_ratio=\"1:1\",\n",
        "    safety_filter_level=\"block_some\",\n",
        "    person_generation=\"allow_adult\",\n",
        ")\n",
        "\n",
        "display_image(images[0])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xGneutCN-mW5"
      },
      "source": [
        "### Add image watermark\n",
        "\n",
        "By default, a digital watermark, or [SynthID](https://deepmind.google/technologies/synthid/), is added to Imagen 3 images. If you would like to explicitly set the watermark to True, you can do so with the `add_watermark` parameter. You can also [verify a watermarked image](https://cloud.google.com/vertex-ai/generative-ai/docs/image/generate-images#watermark)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XtyEx1tYhB8h"
      },
      "outputs": [],
      "source": [
        "prompt = \"\"\"\n",
        "Generate a photo of an old fashioned ice cream shop with white and pastel pink on the outside,\n",
        "located on the street in a sunny beach town,\n",
        "'the sweet stuff' displayed on the storefront with illustrations of pink and blue ice cream cones\n",
        "\"\"\"\n",
        "\n",
        "images = generation_model.generate_images(\n",
        "    prompt=prompt,\n",
        "    number_of_images=1,\n",
        "    aspect_ratio=\"1:1\",\n",
        "    safety_filter_level=\"block_some\",\n",
        "    person_generation=\"allow_adult\",\n",
        "    add_watermark=True,\n",
        ")\n",
        "\n",
        "display_image(images[0])"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "imagen3_image_generation.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}