Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull Imagen feature #285

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/eleven-shoes-raise.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@google/generative-ai": minor
---

Add imagen feature.
38 changes: 38 additions & 0 deletions common/api-review/generative-ai.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -448,6 +448,11 @@ export interface GenerateContentStreamResult {
stream: AsyncGenerator<EnhancedGenerateContentResponse>;
}

// @public
export interface GeneratedImage {
imageBytes?: string;
}

// @public
export interface GenerationConfig {
// (undocumented)
Expand Down Expand Up @@ -511,6 +516,7 @@ export class GoogleGenerativeAI {
apiKey: string;
getGenerativeModel(modelParams: ModelParams, requestOptions?: RequestOptions): GenerativeModel;
getGenerativeModelFromCachedContent(cachedContent: CachedContent, modelParams?: Partial<ModelParams>, requestOptions?: RequestOptions): GenerativeModel;
getImageGenerationModel(modelParams: ModelParams, requestOptions?: RequestOptions): ImageGenerationModel;
}

// @public
Expand Down Expand Up @@ -627,6 +633,38 @@ export enum HarmProbability {
NEGLIGIBLE = "NEGLIGIBLE"
}

// @public
export class ImageGenerationModel {
constructor(apiKey: string, modelParams: ModelParams, _requestOptions?: RequestOptions);
// (undocumented)
apiKey: string;
generateImages(request: ImageGenerationRequest | string, requestOptions?: SingleRequestOptions): Promise<ImageGenerationResponse>;
// (undocumented)
model: string;
}

// @public
export interface ImageGenerationRequest {
aspectRatio?: "1:1" | "9:16" | "16:9" | "4:3" | "3:4";
compressionQuality?: number;
guidanceScale?: number;
height?: number;
language?: string;
negativePrompt?: string;
numberOfImages?: number;
outputMimeType?: "image/png" | "image/jpeg";
personGeneration?: "dont_allow" | "allow_adult";
prompt: string;
safetyFilterLevel?: "block_low_and_above" | "block_medium_and_above" | "block_only_high";
width?: number;
}

// @public
export interface ImageGenerationResponse {
// (undocumented)
images: GeneratedImage[];
}

// @public
export interface InlineDataPart {
// (undocumented)
Expand Down
13 changes: 13 additions & 0 deletions docs/reference/main/generative-ai.generatedimage.imagebytes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [@google/generative-ai](./generative-ai.md) &gt; [GeneratedImage](./generative-ai.generatedimage.md) &gt; [imageBytes](./generative-ai.generatedimage.imagebytes.md)

## GeneratedImage.imageBytes property

Image bytes.

**Signature:**

```typescript
imageBytes?: string;
```
20 changes: 20 additions & 0 deletions docs/reference/main/generative-ai.generatedimage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [@google/generative-ai](./generative-ai.md) &gt; [GeneratedImage](./generative-ai.generatedimage.md)

## GeneratedImage interface

Generated images. It will be returned in response.

**Signature:**

```typescript
export interface GeneratedImage
```

## Properties

| Property | Modifiers | Type | Description |
| --- | --- | --- | --- |
| [imageBytes?](./generative-ai.generatedimage.imagebytes.md) | | string | _(Optional)_ Image bytes. |

Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [@google/generative-ai](./generative-ai.md) &gt; [GoogleGenerativeAI](./generative-ai.googlegenerativeai.md) &gt; [getImageGenerationModel](./generative-ai.googlegenerativeai.getimagegenerationmodel.md)

## GoogleGenerativeAI.getImageGenerationModel() method

Gets a [ImageGenerationModel](./generative-ai.imagegenerationmodel.md) instance for the provided model name.

**Signature:**

```typescript
getImageGenerationModel(modelParams: ModelParams, requestOptions?: RequestOptions): ImageGenerationModel;
```

## Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| modelParams | [ModelParams](./generative-ai.modelparams.md) | |
| requestOptions | [RequestOptions](./generative-ai.requestoptions.md) | _(Optional)_ |

**Returns:**

[ImageGenerationModel](./generative-ai.imagegenerationmodel.md)

1 change: 1 addition & 0 deletions docs/reference/main/generative-ai.googlegenerativeai.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,5 @@ export declare class GoogleGenerativeAI
| --- | --- | --- |
| [getGenerativeModel(modelParams, requestOptions)](./generative-ai.googlegenerativeai.getgenerativemodel.md) | | Gets a [GenerativeModel](./generative-ai.generativemodel.md) instance for the provided model name. |
| [getGenerativeModelFromCachedContent(cachedContent, modelParams, requestOptions)](./generative-ai.googlegenerativeai.getgenerativemodelfromcachedcontent.md) | | Creates a [GenerativeModel](./generative-ai.generativemodel.md) instance from provided content cache. |
| [getImageGenerationModel(modelParams, requestOptions)](./generative-ai.googlegenerativeai.getimagegenerationmodel.md) | | Gets a [ImageGenerationModel](./generative-ai.imagegenerationmodel.md) instance for the provided model name. |

Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [@google/generative-ai](./generative-ai.md) &gt; [ImageGenerationModel](./generative-ai.imagegenerationmodel.md) &gt; [(constructor)](./generative-ai.imagegenerationmodel._constructor_.md)

## ImageGenerationModel.(constructor)

Constructs a new instance of the `ImageGenerationModel` class

**Signature:**

```typescript
constructor(apiKey: string, modelParams: ModelParams, _requestOptions?: RequestOptions);
```

## Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| apiKey | string | |
| modelParams | [ModelParams](./generative-ai.modelparams.md) | |
| \_requestOptions | [RequestOptions](./generative-ai.requestoptions.md) | _(Optional)_ |

11 changes: 11 additions & 0 deletions docs/reference/main/generative-ai.imagegenerationmodel.apikey.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [@google/generative-ai](./generative-ai.md) &gt; [ImageGenerationModel](./generative-ai.imagegenerationmodel.md) &gt; [apiKey](./generative-ai.imagegenerationmodel.apikey.md)

## ImageGenerationModel.apiKey property

**Signature:**

```typescript
apiKey: string;
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [@google/generative-ai](./generative-ai.md) &gt; [ImageGenerationModel](./generative-ai.imagegenerationmodel.md) &gt; [generateImages](./generative-ai.imagegenerationmodel.generateimages.md)

## ImageGenerationModel.generateImages() method

Makes a single non-streaming call to the model and returns an object containing a single [ImageGenerationResponse](./generative-ai.imagegenerationresponse.md)<!-- -->.

Inside the response there will be generated pictures specified by numberOfImages in [ImageGenerationRequest](./generative-ai.imagegenerationrequest.md)

Fields set in the optional [SingleRequestOptions](./generative-ai.singlerequestoptions.md) parameter will take precedence over the [RequestOptions](./generative-ai.requestoptions.md) values provided to [GoogleGenerativeAI.getImageGenerationModel()](./generative-ai.googlegenerativeai.getimagegenerationmodel.md)<!-- -->.

**Signature:**

```typescript
generateImages(request: ImageGenerationRequest | string, requestOptions?: SingleRequestOptions): Promise<ImageGenerationResponse>;
```

## Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| request | [ImageGenerationRequest](./generative-ai.imagegenerationrequest.md) \| string | |
| requestOptions | [SingleRequestOptions](./generative-ai.singlerequestoptions.md) | _(Optional)_ |

**Returns:**

Promise&lt;[ImageGenerationResponse](./generative-ai.imagegenerationresponse.md)<!-- -->&gt;

33 changes: 33 additions & 0 deletions docs/reference/main/generative-ai.imagegenerationmodel.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [@google/generative-ai](./generative-ai.md) &gt; [ImageGenerationModel](./generative-ai.imagegenerationmodel.md)

## ImageGenerationModel class

Class for generative model APIs.

**Signature:**

```typescript
export declare class ImageGenerationModel
```

## Constructors

| Constructor | Modifiers | Description |
| --- | --- | --- |
| [(constructor)(apiKey, modelParams, \_requestOptions)](./generative-ai.imagegenerationmodel._constructor_.md) | | Constructs a new instance of the <code>ImageGenerationModel</code> class |

## Properties

| Property | Modifiers | Type | Description |
| --- | --- | --- | --- |
| [apiKey](./generative-ai.imagegenerationmodel.apikey.md) | | string | |
| [model](./generative-ai.imagegenerationmodel.model.md) | | string | |

## Methods

| Method | Modifiers | Description |
| --- | --- | --- |
| [generateImages(request, requestOptions)](./generative-ai.imagegenerationmodel.generateimages.md) | | <p>Makes a single non-streaming call to the model and returns an object containing a single [ImageGenerationResponse](./generative-ai.imagegenerationresponse.md)<!-- -->.</p><p>Inside the response there will be generated pictures specified by numberOfImages in [ImageGenerationRequest](./generative-ai.imagegenerationrequest.md)</p><p>Fields set in the optional [SingleRequestOptions](./generative-ai.singlerequestoptions.md) parameter will take precedence over the [RequestOptions](./generative-ai.requestoptions.md) values provided to [GoogleGenerativeAI.getImageGenerationModel()](./generative-ai.googlegenerativeai.getimagegenerationmodel.md)<!-- -->.</p> |

11 changes: 11 additions & 0 deletions docs/reference/main/generative-ai.imagegenerationmodel.model.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [@google/generative-ai](./generative-ai.md) &gt; [ImageGenerationModel](./generative-ai.imagegenerationmodel.md) &gt; [model](./generative-ai.imagegenerationmodel.model.md)

## ImageGenerationModel.model property

**Signature:**

```typescript
model: string;
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [@google/generative-ai](./generative-ai.md) &gt; [ImageGenerationRequest](./generative-ai.imagegenerationrequest.md) &gt; [aspectRatio](./generative-ai.imagegenerationrequest.aspectratio.md)

## ImageGenerationRequest.aspectRatio property

Changes the aspect ratio of the generated image. Supported values are: \* "1:1" : 1:1 aspect ratio \* "9:16" : 9:16 aspect ratio \* "16:9" : 16:9 aspect ratio \* "4:3" : 4:3 aspect ratio \* "3:4" : 3:4 aspect\_ratio

**Signature:**

```typescript
aspectRatio?: "1:1" | "9:16" | "16:9" | "4:3" | "3:4";
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [@google/generative-ai](./generative-ai.md) &gt; [ImageGenerationRequest](./generative-ai.imagegenerationrequest.md) &gt; [compressionQuality](./generative-ai.imagegenerationrequest.compressionquality.md)

## ImageGenerationRequest.compressionQuality property

Level of compression if the output mime type is selected to be image/jpeg. Float between 0 to 100

**Signature:**

```typescript
compressionQuality?: number;
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [@google/generative-ai](./generative-ai.md) &gt; [ImageGenerationRequest](./generative-ai.imagegenerationrequest.md) &gt; [guidanceScale](./generative-ai.imagegenerationrequest.guidancescale.md)

## ImageGenerationRequest.guidanceScale property

Controls the strength of the prompt. Suggested values are: \* 0-9 (low strength) \* 10-20 (medium strength) \* 21+ (high strength)

**Signature:**

```typescript
guidanceScale?: number;
```
13 changes: 13 additions & 0 deletions docs/reference/main/generative-ai.imagegenerationrequest.height.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [@google/generative-ai](./generative-ai.md) &gt; [ImageGenerationRequest](./generative-ai.imagegenerationrequest.md) &gt; [height](./generative-ai.imagegenerationrequest.height.md)

## ImageGenerationRequest.height property

Height of the image. One of the Width/Height sizes must be 256 or 1024.

**Signature:**

```typescript
height?: number;
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [@google/generative-ai](./generative-ai.md) &gt; [ImageGenerationRequest](./generative-ai.imagegenerationrequest.md) &gt; [language](./generative-ai.imagegenerationrequest.language.md)

## ImageGenerationRequest.language property

Language of the text prompt for the image. Default: None. Supported values are `"en"` for English, `"hi"` for Hindi, `"ja"` for Japanese, `"ko"` for Korean, and `"auto"` for automatic language detection.

**Signature:**

```typescript
language?: string;
```
31 changes: 31 additions & 0 deletions docs/reference/main/generative-ai.imagegenerationrequest.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [@google/generative-ai](./generative-ai.md) &gt; [ImageGenerationRequest](./generative-ai.imagegenerationrequest.md)

## ImageGenerationRequest interface

Request message for generating image.

**Signature:**

```typescript
export interface ImageGenerationRequest
```

## Properties

| Property | Modifiers | Type | Description |
| --- | --- | --- | --- |
| [aspectRatio?](./generative-ai.imagegenerationrequest.aspectratio.md) | | "1:1" \| "9:16" \| "16:9" \| "4:3" \| "3:4" | _(Optional)_ Changes the aspect ratio of the generated image. Supported values are: \* "1:1" : 1:1 aspect ratio \* "9:16" : 9:16 aspect ratio \* "16:9" : 16:9 aspect ratio \* "4:3" : 4:3 aspect ratio \* "3:4" : 3:4 aspect\_ratio |
| [compressionQuality?](./generative-ai.imagegenerationrequest.compressionquality.md) | | number | _(Optional)_ Level of compression if the output mime type is selected to be image/jpeg. Float between 0 to 100 |
| [guidanceScale?](./generative-ai.imagegenerationrequest.guidancescale.md) | | number | _(Optional)_ Controls the strength of the prompt. Suggested values are: \* 0-9 (low strength) \* 10-20 (medium strength) \* 21+ (high strength) |
| [height?](./generative-ai.imagegenerationrequest.height.md) | | number | _(Optional)_ Height of the image. One of the Width/Height sizes must be 256 or 1024. |
| [language?](./generative-ai.imagegenerationrequest.language.md) | | string | _(Optional)_ Language of the text prompt for the image. Default: None. Supported values are <code>&quot;en&quot;</code> for English, <code>&quot;hi&quot;</code> for Hindi, <code>&quot;ja&quot;</code> for Japanese, <code>&quot;ko&quot;</code> for Korean, and <code>&quot;auto&quot;</code> for automatic language detection. |
| [negativePrompt?](./generative-ai.imagegenerationrequest.negativeprompt.md) | | string | _(Optional)_ A description of what you want to omit in the generated images. |
| [numberOfImages?](./generative-ai.imagegenerationrequest.numberofimages.md) | | number | _(Optional)_ Number of images to generate. Range: 1..4. |
| [outputMimeType?](./generative-ai.imagegenerationrequest.outputmimetype.md) | | "image/png" \| "image/jpeg" | _(Optional)_ Which image format should the output be saved as. Supported values: \* image/png: Save as a PNG image \* image/jpeg: Save as a JPEG image |
| [personGeneration?](./generative-ai.imagegenerationrequest.persongeneration.md) | | "dont\_allow" \| "allow\_adult" | _(Optional)_ Allow generation of people by the model Supported values are: \* "dont\_allow" : Block generation of people \* "allow\_adult" : Generate adults, but not children |
| [prompt](./generative-ai.imagegenerationrequest.prompt.md) | | string | Text prompt for the image. |
| [safetyFilterLevel?](./generative-ai.imagegenerationrequest.safetyfilterlevel.md) | | "block\_low\_and\_above" \| "block\_medium\_and\_above" \| "block\_only\_high" | _(Optional)_ Adds a filter level to Safety filtering. Supported values are: \* "block\_most" : Strongest filtering level, most strict blocking \* "block\_some" : Block some problematic prompts and responses \* "block\_few" : Block fewer problematic prompts and responses |
| [width?](./generative-ai.imagegenerationrequest.width.md) | | number | _(Optional)_ Width of the image. One of the Width/Height sizes must be 256 or 1024. |

Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [@google/generative-ai](./generative-ai.md) &gt; [ImageGenerationRequest](./generative-ai.imagegenerationrequest.md) &gt; [negativePrompt](./generative-ai.imagegenerationrequest.negativeprompt.md)

## ImageGenerationRequest.negativePrompt property

A description of what you want to omit in the generated images.

**Signature:**

```typescript
negativePrompt?: string;
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [@google/generative-ai](./generative-ai.md) &gt; [ImageGenerationRequest](./generative-ai.imagegenerationrequest.md) &gt; [numberOfImages](./generative-ai.imagegenerationrequest.numberofimages.md)

## ImageGenerationRequest.numberOfImages property

Number of images to generate. Range: 1..4.

**Signature:**

```typescript
numberOfImages?: number;
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [@google/generative-ai](./generative-ai.md) &gt; [ImageGenerationRequest](./generative-ai.imagegenerationrequest.md) &gt; [outputMimeType](./generative-ai.imagegenerationrequest.outputmimetype.md)

## ImageGenerationRequest.outputMimeType property

Which image format should the output be saved as. Supported values: \* image/png: Save as a PNG image \* image/jpeg: Save as a JPEG image

**Signature:**

```typescript
outputMimeType?: "image/png" | "image/jpeg";
```
Loading
Loading