Skip to content

Commit

Permalink
Add OpenAI Gemini Example for VertexAI Notebook (#3290)
Browse files Browse the repository at this point in the history 
* add openai-gemini example

* fix exec numbering

* improve isntructions

* fix br  tag

* mention roles/aiplatform.user and fix markdown reference

* remove mentioning the editor role, and only use the Vertex AI User role

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
  • Loading branch information
@luxzoli @sonichi @victordibia
2 people authored and victordibia committed Aug 28, 2024
1 parent a6639f2 commit 57f3de3
Showing 1 changed file with 138 additions and 7 deletions.
145 changes: 138 additions & 7 deletions website/docs/topics/non-openai-models/cloud-gemini_vertexai.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,12 @@
"<img src=\"https://github.com/microsoft/autogen/blob/main/website/static/img/create_gcp_svc.png?raw=true\" width=\"1000\" />\n",
"</div>\n",
"\n",
"For the sake of simplicity we will assign the Editor role to our service account for autogen on our Autogen-with-Gemini Google Cloud project.\n",
"Next we assign the [Vertex AI User](https://cloud.google.com/vertex-ai/docs/general/access-control#aiplatform.user) for the service account. This can be done in the [Google Cloud console](https://console.cloud.google.com/iam-admin/iam?project=autogen-with-gemini) in our `autogen-with-gemini` project.<br/>\n",
"Alternatively, we can also grant the [Vertex AI User](https://cloud.google.com/vertex-ai/docs/general/access-control#aiplatform.user) role by running a command using the gcloud CLI, for example in [Cloud Shell](https://shell.cloud.google.com/cloudshell):\n",
"```bash\n",
"gcloud projects add-iam-policy-binding autogen-with-gemini \\\n",
" --member=serviceAccount:autogen@autogen-with-gemini.iam.gserviceaccount.com --role roles/aiplatform.user\n",
"```\n",
"\n",
"* Under IAM & Admin > Service Account select the newly created service accounts, and click the option \"Manage keys\" among the items. \n",
"* From the \"ADD KEY\" dropdown select \"Create new key\" and select the JSON format and click CREATE.\n",
Expand All @@ -83,23 +88,23 @@
"Additionally, AutoGen also supports authentication using `Credentials` objects in Python with the [google-auth library](https://google-auth.readthedocs.io/), which enables even more flexibility.<br/>\n",
"For example, we can even use impersonated credentials.\n",
"\n",
"#### Use Service Account Keyfile\n",
"#### <a id='use_svc_keyfile'></a>Use Service Account Keyfile\n",
"\n",
"The Google Cloud service account can be specified by setting the `GOOGLE_APPLICATION_CREDENTIALS` environment variable to the path to the JSON key file of the service account. <br/>\n",
"\n",
"We could even just directly set the environment variable, or we can add the `\"google_application_credentials\"` key with the respective value for our model in the OAI_CONFIG_LIST.\n",
"\n",
"#### Use the Google Default Credentials\n",
"\n",
"If you are using [Cloud Shell](https://shell.cloud.google.com/cloudshell) or [Cloud Shell editor](https://shell.cloud.google.com/cloudshell/editor) in Google Cloud, <br/> then you are already authenticated. If you have the Google Cloud SDK installed locally, <br/> then you can login by running `gcloud auth login` in the command line. \n",
"If you are using [Cloud Shell](https://shell.cloud.google.com/cloudshell) or [Cloud Shell editor](https://shell.cloud.google.com/cloudshell/editor) in Google Cloud, <br/> then you are already authenticated. If you have the Google Cloud SDK installed locally, <br/> then you can login by running `gcloud auth application-default login` in the command line. \n",
"\n",
"Detailed instructions for installing the Google Cloud SDK can be found [here](https://cloud.google.com/sdk/docs/install).\n",
"\n",
"#### Authentication with the Google Auth Library for Python\n",
"\n",
"The google-auth library supports a wide range of authentication scenarios, and you can simply pass a previously created `Credentials` object to the `llm_config`.<br/>\n",
"The [official documentation](https://google-auth.readthedocs.io/) of the Python package provides a detailed overview of the supported methods and usage examples.<br/>\n",
"If you are already authenticated, like in [Cloud Shell](https://shell.cloud.google.com/cloudshell), or after running the `gcloud auth login` command in a CLI, then the `google.auth.default()` Python method will automatically return your currently active credentials."
"If you are already authenticated, like in [Cloud Shell](https://shell.cloud.google.com/cloudshell), or after running the `gcloud auth application-default login` command in a CLI, then the `google.auth.default()` Python method will automatically return your currently active credentials."
]
},
{
Expand Down Expand Up @@ -307,13 +312,13 @@
"gcloud auth application-default login\n",
"gcloud config set project autogen-with-gemini\n",
"```\n",
"The `GOOGLE_APPLICATION_CREDENTIALS` environment variable is a path to our service account JSON keyfile, as described in the [Use Service Account Keyfile](#Use Service Account Keyfile) section above.<br/>\n",
"The `GOOGLE_APPLICATION_CREDENTIALS` environment variable is a path to our service account JSON keyfile, as described in the [Use Service Account Keyfile](#use_svc_keyfile) section above.<br/>\n",
"We also need to set the Google cloud project, which is `autogen-with-gemini` in this example.<br/><br/>\n",
"\n",
"Note, we could also run `gcloud auth login` in case we wish to use our personal Google account instead of a service account.\n",
"Note, we could also run `gcloud auth application-default login` to use our personal Google account instead of a service account.\n",
"In this case we need to run the following commands:\n",
"```bash\n",
"gcloud auth login\n",
"gcloud gcloud auth application-default login\n",
"gcloud config set project autogen-with-gemini\n",
"```"
]
Expand Down Expand Up @@ -390,6 +395,132 @@
"<img https://github.com/microsoft/autogen/blob/main/website/static/img/autogen_agentchat.png?raw=true>.\"\"\",\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Use Gemini via the OpenAI Library in Autogen\n",
"Using Gemini via the OpenAI library is also possible once you are already authenticated. <br/>\n",
"Run `gcloud auth application-default login` to set up application default credentials locally for the example below.<br/>\n",
"Also set the Google cloud project on the CLI if you have not done so far: <br/>\n",
"```bash\n",
"gcloud config set project autogen-with-gemini\n",
"```\n",
"The prerequisites are essentially the same as in the example above.<br/>\n",
"\n",
"You can read more on the topic in the [official Google docs](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/call-gemini-using-openai-library).\n",
"<br/> A list of currently supported models can also be found in the [docs](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/call-gemini-using-openai-library#supported_models)\n",
"<br/>\n",
"<br/>\n",
"Note, that you will need to refresh your token regularly, by default every 1 hour."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"import google.auth\n",
"\n",
"scopes = [\"https://www.googleapis.com/auth/cloud-platform\"]\n",
"creds, project = google.auth.default(scopes)\n",
"auth_req = google.auth.transport.requests.Request()\n",
"creds.refresh(auth_req)\n",
"location = \"us-west1\"\n",
"prompt_price_per_1k = (\n",
" 0.000125 # For more up-to-date prices see https://cloud.google.com/vertex-ai/generative-ai/pricing\n",
")\n",
"completion_token_price_per_1k = (\n",
" 0.000375 # For more up-to-date prices see https://cloud.google.com/vertex-ai/generative-ai/pricing\n",
")\n",
"\n",
"openai_gemini_config = [\n",
" {\n",
" \"model\": \"google/gemini-1.5-pro-001\",\n",
" \"api_type\": \"openai\",\n",
" \"base_url\": f\"https://{location}-aiplatform.googleapis.com/v1beta1/projects/{project}/locations/{location}/endpoints/openapi\",\n",
" \"api_key\": creds.token,\n",
" \"price\": [prompt_price_per_1k, completion_token_price_per_1k],\n",
" }\n",
"]"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[33muser_proxy\u001b[0m (to assistant):\n",
"\n",
"\n",
" Compute the integral of the function f(x)=x^3 on the interval 0 to 10 using a Python script,\n",
" which returns the value of the definite integral.\n",
"\n",
"--------------------------------------------------------------------------------\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[33massistant\u001b[0m (to user_proxy):\n",
"\n",
"```python\n",
"# filename: integral.py\n",
"def integrate_x_cubed(a, b):\n",
" \"\"\"\n",
" This function calculates the definite integral of x^3 from a to b.\n",
"\n",
" Args:\n",
" a: The lower limit of integration.\n",
" b: The upper limit of integration.\n",
"\n",
" Returns:\n",
" The value of the definite integral.\n",
" \"\"\"\n",
" return (b**4 - a**4) / 4\n",
"\n",
"# Calculate the integral of x^3 from 0 to 10\n",
"result = integrate_x_cubed(0, 10)\n",
"\n",
"# Print the result\n",
"print(result)\n",
"```\n",
"\n",
"This script defines a function `integrate_x_cubed` that takes the lower and upper limits of integration as arguments and returns the definite integral of x^3 using the power rule of integration. The script then calls this function with the limits 0 and 10 and prints the result.\n",
"\n",
"Execute the script `python integral.py`, you should get the result: `2500.0`.\n",
"\n",
"TERMINATE\n",
"\n",
"\n",
"--------------------------------------------------------------------------------\n"
]
}
],
"source": [
"assistant = AssistantAgent(\"assistant\", llm_config={\"config_list\": openai_gemini_config}, max_consecutive_auto_reply=3)\n",
"\n",
"user_proxy = UserProxyAgent(\n",
" \"user_proxy\",\n",
" code_execution_config={\"work_dir\": \"coding\", \"use_docker\": False},\n",
" human_input_mode=\"NEVER\",\n",
" is_termination_msg=lambda x: content_str(x.get(\"content\")).find(\"TERMINATE\") >= 0,\n",
")\n",
"\n",
"result = user_proxy.initiate_chat(\n",
" assistant,\n",
" message=\"\"\"\n",
" Compute the integral of the function f(x)=x^3 on the interval 0 to 10 using a Python script,\n",
" which returns the value of the definite integral.\"\"\",\n",
")"
]
}
],
"metadata": {
Expand Down

0 comments on commit 57f3de3

Please sign in to comment.