adding gpt-image-1 support

[pelikhan]

---

• 🌟 Added support for the openai:gpt-image-1 model in image generation workflows:

  • Included openai:gpt-image-1 in the ModelImageGenerationType public API.
  • Updated the sample script to use openai:gpt-image-1 as the model in an example.

• 🧹 Dynamic parameter adjustments for GPT-based models:

  • Removed style and response_format parameters when models matching gpt-* are used.

• 🛠️ Enhanced debugging and traceability:

  • Added debug logging for API responses, including status code and response body.
  • Improved trace details to capture revised prompts when available.

AI-generated content by pr-describe may be incorrect. Use reactions to eval.

[github-actions]

LGTM 🚀

AI-generated content by pr-review may be incorrect. Use reactions to eval.

[pelikhan]

Blocked on model access.

[Copilot]

Pull Request Overview

This PR adds support for the new openai:gpt-image-1 model in image generation workflows while dynamically removing incompatible parameters and enhancing debugging details.

• Added a new model parameter in the sample script for image generation.
• Introduced conditional removal of parameters and additional debug logging in the OpenAI image generation module.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
packages/sample/genaisrc/image-gen.genai.mjs	Added model parameter for openai:gpt-image-1 in image generation
packages/core/src/openai.ts	Removed style and response_format for GPT-based models and added debug logging

[Copilot]

The regex /^gpt-/i does not match the new model string 'openai:gpt-image-1'. Consider updating the condition to correctly identify GPT-based models, for example by checking if the model string includes 'gpt-'.

Suggested change

	if (/^gpt-/i.test(model)) {
	if (/\bgpt-/i.test(model)) {

[Copilot]

Pull Request Overview

This PR adds support for the openai:gpt-image-1 model in image generation workflows while also applying dynamic parameter adjustments for GPT-based models and enhancing debugging details.

• Updated the sample script to include the new image generation model.
• Modified core image generation logic to adjust parameters and improve debugging output.
• Changed the image debug logging import and broadened the accepted types for the quality parameter.

Reviewed Changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated no comments.

File	Description
packages/sample/genaisrc/image-gen.genai.mjs	Updated sample script to iterate over supported models including openai:gpt-image-1.
packages/core/src/openai.ts	Adjusted parameter handling for various models and added additional debugging logging.
packages/core/src/image.ts	Updated debug import to use genaiscriptDebug for consistency.
packages/core/src/chat.ts	Changed quality type from a specific "hd" to a generic string to support dynamic values.

Files not reviewed (1)

• .vscode/settings.json: Language not supported

[github-actions]

diff --git a/packages/core/src/chat.ts b/packages/core/src/chat.ts
index e0cbfb42..25dc40dc 100644
--- a/packages/core/src/chat.ts
+++ b/packages/core/src/chat.ts
@@ -167,7 +167,7 @@ export type SpeechFunction = (
 export type CreateImageRequest = {
     model: string
     prompt: string
-    quality?: "hd"
+    quality?: string
     size?: string
     style?: string
 }
diff --git a/packages/core/src/image.ts b/packages/core/src/image.ts
index d2bcd8ca..c3b4783b 100644
--- a/packages/core/src/image.ts
+++ b/packages/core/src/image.ts
@@ -1,6 +1,3 @@
-import debug from "debug"
-const dbg = debug("genaiscript:image")
-
 // Import necessary functions and types from other modules
 import { resolveBufferLike } from "./bufferlike"
 import {
@@ -16,6 +13,8 @@ import pLimit from "p-limit"
 import { CancellationOptions, checkCancelled } from "./cancellation"
 import { wrapColor, wrapRgbColor } from "./consolecolor"
 import { assert } from "console"
+import { genaiscriptDebug } from "./debug"
+const dbg = genaiscriptDebug("image")
 
 async function prepare(
     url: BufferLike,
diff --git a/packages/core/src/openai.ts b/packages/core/src/openai.ts
index 7a10492f..1796db1d 100644
--- a/packages/core/src/openai.ts
+++ b/packages/core/src/openai.ts
@@ -762,15 +762,56 @@ export async function OpenAIImageGeneration(
     const { trace } = options || {}
     let url = `${cfg.base}/images/generations`
 
-    const body = {
+    const isDallE = /^dall-e/i.test(model)
+    const isDallE2 = /^dall-e-3/i.test(model)
+    const isDallE3 = /^dall-e-3/i.test(model)
+    const isGpt = /^gpt-image/i.test(model)
+
+    const body: any = {
         model,
         prompt,
         size,
         quality,
         style,
-        response_format: "b64_json",
         ...rest,
     }
+
+    // auto is the default quality, so always delete it
+    if (body.quality === "auto" || isDallE2) delete body.quality
+    if (isDallE3) {
+        if (body.quality === "high") body.quality = "hd"
+        else delete body.quality
+    }
+    if (isGpt && body.quality === "hd") body.quality = "high"
+    if (!isDallE3) delete body.style
+    if (isDallE) body.response_format = "b64_json"
+
+    if (isDallE3) {
+        if (body.size === "portrait") body.size = "1024x1792"
+        else if (body.size === "landscape") body.size = "1792x1024"
+        else if (body.size === "square") body.size = "1024x1024"
+    } else if (isDallE2) {
+        if (
+            body.size === "portrait" ||
+            body.size === "landscape" ||
+            body.size === "square"
+        )
+            body.size = "1024x1024"
+    } else if (isGpt) {
+        if (body.size === "portrait") body.size = "1024x1536"
+        else if (body.size === "landscape") body.size = "1536x1024"
+        else if (body.size === "square") body.size = "1024x1024"
+    }
+
+    if (body.size === "auto") delete body.size
+
+    dbg({
+        quality: body.quality,
+        style: body.style,
+        response_format: body.response_format,
+        size: body.size,
+    })
+
     if (cfg.type === "azure") {
         const version = cfg.version || AZURE_OPENAI_API_VERSION
         trace?.itemValue(`version`, version)
@@ -797,6 +838,7 @@ export async function OpenAIImageGeneration(
         trace?.itemValue(`url`, `[${url}](${url})`)
         traceFetchPost(trace, url, freq.headers, body)
         const res = await fetch(url, freq as any)
+        dbg(`response: %d %s`, res.status, res.statusText)
         trace?.itemValue(`status`, `${res.status} ${res.statusText}`)
         if (!res.ok)
             return {
@@ -804,6 +846,7 @@ export async function OpenAIImageGeneration(
                 error: (await res.json())?.error || res.statusText,
             }
         const j = await res.json()
+        dbg(`%O`, j)
         const revisedPrompt = j.data[0]?.revised_prompt
         if (revisedPrompt)
             trace?.details(`📷 revised prompt`, j.data[0].revised_prompt)
diff --git a/packages/core/src/types/prompt_template.d.ts b/packages/core/src/types/prompt_template.d.ts
index d6713931..a30558ad 100644
--- a/packages/core/src/types/prompt_template.d.ts
+++ b/packages/core/src/types/prompt_template.d.ts
@@ -307,7 +307,7 @@ type ModelVisionType = OptionsOrString<
 >
 
 type ModelImageGenerationType = OptionsOrString<
-    "openai:dall-e-2" | "openai:dall-e-3"
+    "openai:gpt-image-1" | "openai:dall-e-2" | "openai:dall-e-3"
 >
 
 type ModelProviderType = OptionsOrString<
@@ -4380,10 +4380,35 @@ type TranscriptionModelType = OptionsOrString<
 
 interface ImageGenerationOptions extends ImageTransformOptions {
     model?: OptionsOrString<ModelImageGenerationType>
-    quality?: "hd"
+    /**
+     * The quality of the image that will be generated.
+     * auto (default value) will automatically select the best quality for the given model.
+     * high, medium and low are supported for gpt-image-1.
+     * high is supported for dall-e-3.
+     * dall-e-2 ignores this flag
+     */
+    quality?: "auto" | "low" | "medium" | "high"
+    /**
+     * Image size.
+     * For gpt-image-1: 1024x1024, 1536x1024 (landscape), 1024x1536 (portrait), or auto (default value)
+     * For dall-e: 256x256, 512x512, or 1024x1024 for dall-e-2, and one of 1024x1024, 1792x1024.
+     */
     size?: OptionsOrString<
-        "256x256" | "512x512" | "1024x1024" | "1024x1792" | "1792x1024"
+        | "auto"
+        | "landscape"
+        | "portrait"
+        | "square"
+        | "1536x1024"
+        | "1024x1536"
+        | "256x256"
+        | "512x512"
+        | "1024x1024"
+        | "1024x1792"
+        | "1792x1024"
     >
+    /**
+     * Only used for DALL-E 3
+     */
     style?: OptionsOrString<"vivid" | "natural">
 }
 
diff --git a/packages/sample/genaisrc/image-gen.genai.mjs b/packages/sample/genaisrc/image-gen.genai.mjs
index 262c2776..a6e9cfb3 100644
--- a/packages/sample/genaisrc/image-gen.genai.mjs
+++ b/packages/sample/genaisrc/image-gen.genai.mjs
@@ -1,6 +1,19 @@
-const { image, revisedPrompt } = await generateImage(
-    `a cute cat. only one. iconic, high details. 8-bit resolution.`,
-    { maxWidth: 400, mime: "image/png" }
-)
-env.output.image(image.filename)
-env.output.fence(revisedPrompt)
+for (const model of [
+    "openai:dall-e-2",
+    "openai:dall-e-3",
+    "openai:gpt-image-1",
+]) {
+    env.output.heading(3, `Model: ${model}`)
+    const { image, revisedPrompt } = await generateImage(
+        `a cute cat. only one. iconic, high details. 8-bit resolution.`,
+        {
+            maxWidth: 400,
+            autoCrop: true,
+            mime: "image/png",
+            model,
+            size: "square",
+        }
+    )
+    env.output.image(image.filename)
+    env.output.fence(revisedPrompt)
+}

filename	kind	gen	genCost	judge	judgeCost	edits	updated
packages/core/src/openai.ts	update	1999	0.005876	1824	0.003654	1	1

AI-generated content by docs may be incorrect. Use reactions to eval.

[Copilot]

Pull Request Overview

This PR adds support for the new openai:gpt-image-1 model along with dynamic parameter adjustments for GPT-based image generation workflows and enhanced debug logging for better traceability.

• Added openai:gpt-image-1 to the public API and sample script
• Removed style and response_format parameters for gpt-based models
• Improved logging details for image generation API responses

Reviewed Changes

Copilot reviewed 12 out of 14 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
packages/sample/genaisrc/image-gen.genai.mjs	Updated sample script to iterate over three models including gpt-image-1, with new parameters (autoCrop, size, model) set.
packages/core/src/openai.ts	Enhanced dynamic request body construction for image generation, including new adjustments for gpt-image-1.
packages/core/src/image.ts	Refactored image processing with updated debug logging and conditional resizing logic.
packages/core/src/chat.ts	Updated the quality parameter’s type signature to a more flexible string to support new models.
docs/public/blog/gpt-image-1.txt	Added a blog post introducing the new gpt-image-1 model and its capabilities.
docs/genaisrc/blog-image.genai.mts	Revised the image generation prompt and parameters in the documentation to use gpt-image-1.

Files not reviewed (2)

• .vscode/settings.json: Language not supported
• docs/src/content/docs/blog/gpt-image-1.mdx: Language not supported

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[github-actions]

Annotations

variable-names

Note

You added new code. Consider writing tests to cover these changes. (docs/genaisrc/blog-image.genai.mts#L10 tests_missing)

no-fix-mes

No issues found.

grumpy-dev

Caution

Removing scale without explanation? Back in my day, we'd document every single change—not just nuke things silently. (docs/genaisrc/blog-image.genai.mts#L10 inefficient_code_removal)

Warning

You removed debugging statements. Fine, but how are we debugging this in production now? Guess I need to remind you what a nightmare debugging becomes without logs. Sigh, amateurs. (packages/core/src/image.ts#L18 debugging_removed)

Caution

Removing standard resolutions like 256x256 and 512x512? Clearly, someone doesn't consider older infrastructure compatibility anymore. (packages/core/src/types/prompt_template.d.ts#L20 unsupported_formats)

stats

Category	Count
Files Modified	7
Lines Added	12
Lines Removed	25
Total Changes	-13

diagrams

graph TD
    A[docs/genaisrc/blog-image.genai.mts] --> B[packages/core/src/image.ts]
    A --> C[docs/public/blog/gpt-image-1.mdx]
    A --> D[packages/core/src/types/prompt_template.d.ts]

Loading

description

Warning

The description of the pull request does not match the actual code changes. Review the PR description to ensure it fully explains the removal of image scaling, debugging code, and supported resolution formats. (docs/genaisrc/blog-image.genai.mts#L10 pr_description_mismatch)

comments

Caution

There is no explanation for what {response_format: "b64_json"} represents. Add a comment to clarify its purpose for future developers. (packages/core/src/openai.ts#L19 missing_comments)

Caution

No description exists for the critical contain() process. Completely unacceptable. Document this properly. (packages/core/src/image.ts#L18 barely_any_comments)

---

Let me summarize: poorly justified changes, crucial pieces of debugging removed, backward compatibility tossed out. Typical AI-chaos. Add tests, properly explain your work, and for the love of all that's holy, leave the debugging in!

AI-generated content by linters may be incorrect. Use reactions to eval.