experimental_generateImage Token Usage

[aayush-kapoor]

Description

The spec change in #10128 allows to expose the number of tokens the model used to generate image.

Support needs to be added for the following providers:

• openai
• fal
• amazon-bedrock
• azure
• deepinfra
• fireworks
• google
• google-vertex
• luma
• replicate
• togetherai
• xai

If you want to work on adding support for any of these providers, please create a sub-issue first based on the provider you're working on.

Go through the API documentation for each of these providers and see how they expose the tokens used

[kevinjosethomas]

Hey! I was hoping to tackle some of these providers, and went through their API documentation to see how they expose usage information.

Outside of OpenAI's API, I couldn't find token usage data in the image generation APIs for any of the following endpoints:

• fal
• azure
• deepinfra
• fireworks
• luma
• replicate
• togetherai
• xai

For amazon-bedrock, I could only find the CountTokens endpoint which wouldn't make sense to use. Similarly, google, provides fixed token counts for specific aspect ratios, but they don't account for input tokens.

Some, like deepinfra, follow the OpenAI API Schema, but I don't believe they provide the usage token metadata regardless. It might be worth running inference to confirm that.

I've linked the documentation pages I referred to for each provider, in case you are able to find something I missed.

[Kartikayy007]

Hey! I was hoping to tackle some of these providers, and went through their API documentation to see how they expose usage information.

Outside of OpenAI's API, I couldn't find token usage data in the image generation APIs for any of the following endpoints:

• fal
• azure
• deepinfra
• fireworks
• luma
• replicate
• togetherai
• xai

For amazon-bedrock, I could only find the CountTokens endpoint which wouldn't make sense to use. Similarly, google, provides fixed token counts for specific aspect ratios, but they don't account for input tokens.

Some, like deepinfra, follow the OpenAI API Schema, but I don't believe they provide the usage token metadata regardless. It might be worth running inference to confirm that.

I've linked the documentation pages I referred to for each provider, in case you are able to find something I missed.

Yea, I agree Deepinfra but does provides cost in the metadata instead of token usage, @aayush-kapoor @gr2m need your suggestions here.