diff --git a/public/__redirects b/public/__redirects index 805e4bb35b5ba9..af679e78384c4e 100644 --- a/public/__redirects +++ b/public/__redirects @@ -146,6 +146,10 @@ /ai-gateway/integration/aig-workers-ai-binding/ /ai-gateway/integrations/aig-workers-ai-binding/ 301 /ai-gateway/integration/ /ai-gateway/integrations/ 301 /ai-gateway/providers/open-router/ /ai-gateway/providers/openrouter/ 301 +/ai-gateway/configuration/websockets-api/ /ai-gateway/websockets-api/ 301 +/ai-gateway/configuration/websockets-api/non-realtime-api/ /ai-gateway/websockets-api/non-realtime-api/ 301 +/ai-gateway/configuration/websockets-api/realtime-api/ /ai-gateway/websockets-api/realtime-api/ 301 + # agents /agents/capabilities/mcp-server/ /agents/model-context-protocol/ 301 diff --git a/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx b/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx index 5c9a63ac90fc26..09d079f5196baf 100644 --- a/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx +++ b/src/content/changelog/ai-gateway/2025-03-20-websockets.mdx @@ -1,37 +1,39 @@ --- -title: AI Gateway launches Realtime WebSockets API -description: AI Gateway now supports end-to-end, client-to-provider WebSockets +title: AI Gateway launches Realtime WebSockets API +description: AI Gateway now supports end-to-end, client-to-provider WebSockets date: 2025-03-21T09:00:00Z --- -We are excited to announce that [AI Gateway](/ai-gateway/) now supports real-time AI interactions with the new [Realtime WebSockets API](/ai-gateway/configuration/websockets-api/realtime-api/). + +We are excited to announce that [AI Gateway](/ai-gateway/) now supports real-time AI interactions with the new [Realtime WebSockets API](/ai-gateway/websockets-api/realtime-api/). This new capability allows developers to establish persistent, low-latency connections between their applications and AI models, enabling natural, real-time conversational AI experiences, including speech-to-speech interactions. The Realtime WebSockets API works with the [OpenAI Realtime API](https://platform.openai.com/docs/guides/realtime#connect-with-websockets), [Google Gemini Live API](https://ai.google.dev/gemini-api/docs/multimodal-live), and supports real-time text and speech interactions with models from [Cartesia](https://docs.cartesia.ai/api-reference/tts/tts), and [ElevenLabs](https://elevenlabs.io/docs/conversational-ai/api-reference/conversational-ai/websocket). Here's how you can connect AI Gateway to [OpenAI's Realtime API](https://platform.openai.com/docs/guides/realtime#connect-with-websockets) using WebSockets: + ```javascript title="OpenAI Realtime API example" import WebSocket from "ws"; const url = - "wss://gateway.ai.cloudflare.com/v1///openai?model=gpt-4o-realtime-preview-2024-12-17"; + "wss://gateway.ai.cloudflare.com/v1///openai?model=gpt-4o-realtime-preview-2024-12-17"; const ws = new WebSocket(url, { - headers: { - "cf-aig-authorization": process.env.CLOUDFLARE_API_KEY, - Authorization: "Bearer " + process.env.OPENAI_API_KEY, - "OpenAI-Beta": "realtime=v1", - }, + headers: { + "cf-aig-authorization": process.env.CLOUDFLARE_API_KEY, + Authorization: "Bearer " + process.env.OPENAI_API_KEY, + "OpenAI-Beta": "realtime=v1", + }, }); ws.on("open", () => console.log("Connected to server.")); ws.on("message", (message) => console.log(JSON.parse(message.toString()))); ws.send( - JSON.stringify({ - type: "response.create", - response: { modalities: ["text"], instructions: "Tell me a joke" }, - }), + JSON.stringify({ + type: "response.create", + response: { modalities: ["text"], instructions: "Tell me a joke" }, + }), ); ``` -Get started by checking out the [Realtime WebSockets API](/ai-gateway/configuration/websockets-api/realtime-api/) documentation. +Get started by checking out the [Realtime WebSockets API](/ai-gateway/websockets-api/realtime-api/) documentation. diff --git a/src/content/docs/ai-gateway/get-started.mdx b/src/content/docs/ai-gateway/get-started.mdx index 1a08fb0b33c16c..b27261595dabe7 100644 --- a/src/content/docs/ai-gateway/get-started.mdx +++ b/src/content/docs/ai-gateway/get-started.mdx @@ -32,9 +32,9 @@ When setting up a new gateway, you can choose between an authenticated and unaut Next, connect your AI provider to your gateway. -AI Gateway offers multiple endpoints for each Gateway you create - one endpoint per provider, and one Universal Endpoint. To use AI Gateway, you will need to create your own account with each provider and provide your API key. AI Gateway acts as a proxy for these requests, enabling observability, caching, and more. +AI Gateway offers multiple endpoints for each Gateway you create - one endpoint per provider, and one Universal Endpoint. To use AI Gateway, you will need to create your own account with each provider and provide your API key. AI Gateway acts as a proxy for these requests, enabling observability, caching, and more. -Additionally, AI Gateway has a [WebSockets API](/ai-gateway/configuration/websockets-api/) which provides a single persistent connection, enabling continuous communication. This API supports all AI providers connected to AI Gateway, including those that do not natively support WebSockets. +Additionally, AI Gateway has a [WebSockets API](/ai-gateway/websockets-api/) which provides a single persistent connection, enabling continuous communication. This API supports all AI providers connected to AI Gateway, including those that do not natively support WebSockets. Below is a list of our supported model providers: diff --git a/src/content/docs/ai-gateway/providers/universal.mdx b/src/content/docs/ai-gateway/providers/universal.mdx index 09fe85e847a76c..ea3e391552da67 100644 --- a/src/content/docs/ai-gateway/providers/universal.mdx +++ b/src/content/docs/ai-gateway/providers/universal.mdx @@ -31,7 +31,7 @@ The above will send a request to Workers AI Inference API, if it fails it will p ## WebSockets API -The Universal Endpoint can also be accessed via a [WebSockets API](/ai-gateway/configuration/websockets-api/) which provides a single persistent connection, enabling continuous communication. This API supports all AI providers connected to AI Gateway, including those that do not natively support WebSockets. +The Universal Endpoint can also be accessed via a [WebSockets API](/ai-gateway/websockets-api/) which provides a single persistent connection, enabling continuous communication. This API supports all AI providers connected to AI Gateway, including those that do not natively support WebSockets. ## WebSockets example diff --git a/src/content/docs/ai-gateway/configuration/websockets-api/index.mdx b/src/content/docs/ai-gateway/websockets-api/index.mdx similarity index 66% rename from src/content/docs/ai-gateway/configuration/websockets-api/index.mdx rename to src/content/docs/ai-gateway/websockets-api/index.mdx index a16beb493782d3..f43487e47e35ab 100644 --- a/src/content/docs/ai-gateway/configuration/websockets-api/index.mdx +++ b/src/content/docs/ai-gateway/websockets-api/index.mdx @@ -4,6 +4,7 @@ pcx_content_type: configuration sidebar: group: badge: Beta + order: 5 --- The AI Gateway WebSockets API provides a persistent connection for AI interactions, eliminating repeated handshakes and reducing latency. This API is divided into two categories: @@ -20,16 +21,16 @@ WebSockets are long-lived TCP connections that enable bi-directional, real-time - **Reduced overhead**: Avoid overhead of repeated handshakes and TLS negotiations by maintaining a single, persistent connection. - **Provider compatibility**: Works with all AI providers in AI Gateway. Even if your chosen provider does not support WebSockets, Cloudflare handles it for you, managing the requests to your preferred AI provider. -## **Key differences** +## Key differences -| Feature | Realtime APIs | Non-Realtime APIs | -| :---------------------- | :------------------------------------------------------------------------------------------------------------------------------------ | :----------------------------------------------------------------------------------------------- | -| **Purpose** | Enables real-time, multimodal AI interactions for providers that offer dedicated WebSocket endpoints. | Supports WebSocket-based AI interactions with providers that do not natively support WebSockets. | -| **Use Case** | Streaming responses for voice, video, and live interactions. | Text-based queries and responses, such as LLM requests. | -| **AI Provider Support** | [Limited to providers offering real-time WebSocket APIs.](/ai-gateway/configuration/websockets-api/realtime-api/#supported-providers) | [All AI providers in AI Gateway.](/ai-gateway/providers) | -| **Streaming Support** | Providers natively support real-time data streaming. | AI Gateway handles streaming via WebSockets. | +| Feature | Realtime APIs | Non-Realtime APIs | +| :---------------------- | :---------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------- | +| **Purpose** | Enables real-time, multimodal AI interactions for providers that offer dedicated WebSocket endpoints. | Supports WebSocket-based AI interactions with providers that do not natively support WebSockets. | +| **Use Case** | Streaming responses for voice, video, and live interactions. | Text-based queries and responses, such as LLM requests. | +| **AI Provider Support** | [Limited to providers offering real-time WebSocket APIs.](/ai-gateway/websockets-api/realtime-api/#supported-providers) | [All AI providers in AI Gateway.](/ai-gateway/providers/) | +| **Streaming Support** | Providers natively support real-time data streaming. | AI Gateway handles streaming via WebSockets. | For details on implementation, refer to the next sections: -- [Realtime WebSockets API](/ai-gateway/configuration/websockets-api/realtime-api/) -- [Non-Realtime WebSockets API](/ai-gateway/configuration/websockets-api/non-realtime-api/) +- [Realtime WebSockets API](/ai-gateway/websockets-api/realtime-api/) +- [Non-Realtime WebSockets API](/ai-gateway/websockets-api/non-realtime-api/) diff --git a/src/content/docs/ai-gateway/configuration/websockets-api/non-realtime-api.mdx b/src/content/docs/ai-gateway/websockets-api/non-realtime-api.mdx similarity index 100% rename from src/content/docs/ai-gateway/configuration/websockets-api/non-realtime-api.mdx rename to src/content/docs/ai-gateway/websockets-api/non-realtime-api.mdx diff --git a/src/content/docs/ai-gateway/configuration/websockets-api/realtime-api.mdx b/src/content/docs/ai-gateway/websockets-api/realtime-api.mdx similarity index 100% rename from src/content/docs/ai-gateway/configuration/websockets-api/realtime-api.mdx rename to src/content/docs/ai-gateway/websockets-api/realtime-api.mdx