feat: deployed LLM agent to Functions

docs/services.md

@@ -162,14 +162,57 @@ $ gcloud run jobs execute embeddings --region us-west1
 ## Reddit tool / agent
 
 The [Reddit tool](../services/reddit-tool/) allows the LLM to read [r/travel][subreddit] posts based
-upon a user query. The tool is packaged as a Vertex AI [Reasoning Engine agent][reasoning]. Internally, 
-the tool uses [LangChain][langchain] along with the Vertex AI Python SDK to perform its
-magic.
+upon a user query. The tool is packaged as a Vertex AI [Reasoning Engine agent][reasoning]. 
+Internally,  the tool uses [LangChain][langchain] along with the Vertex AI Python
+SDK to perform its magic.
 
-### Deploy the agent
+**WARNING**: As of writing (2024-11-26), the Vertex AI Reasoning Engine agent
+doesn't work as intended. Instead, the agent is published to Cloud Functions.
 
-**NOTE**: You might need to install `pyenv` first before completing these instructions.
-See [Troubleshooting](./troubleshooting.md) for more details.
+### Test the agent locally (Cloud Functions)
+
+1. Run the Cloud Function locally.
+
+```sh
+functions-framework-python --target get_agent_request
+```
+
+1. Send a request to the app with `curl`.
+
+```sh
+curl --header "Content-Type: application/json" \
+  --request POST \
+  --data '{"query":"I want to go to Crete. Where should I stay?"}' \
+  http://localhost:8080
+```
+
+Deployed location:
+https://reddit-tool-1025771077852.us-west1.run.app
+
+### Deploy the agent (Cloud Functions)
+
+Run the following from the root of the reddit-tool directory.
+
+```sh
+  gcloud functions deploy reddit-tool \
+    --gen2 \
+    --memory=512MB \
+    --timeout=120s \
+    --runtime=python312 \
+    --region=us-west1 \
+    --set-env-vars PROJECT_ID=${PROJECT_ID},BUCKET=${BUCKET} \
+    --source=. \
+    --entry-point=get_agent_request \
+    --trigger-http \
+    --allow-unauthenticated
+```
+
+### Deploy the agent (Reasoning Engine)
+
+**NOTES**: 
++ You might need to install `pyenv` first before completing these instructions.
+  See [Troubleshooting](./troubleshooting.md) for more details.
++ 
 
 1. Create a virtual environment. The virtual environment needs to have Python v3.6 <= x <= v3.11.
 

services/evaluations/cloudbuild.yaml

@@ -3,7 +3,7 @@ steps:
   env:
   - 'DATASET_NAME=myherodotus'
   script: |
-    docker build -t us-west1-docker.pkg.dev/$PROJECT_ID/my-herodotus/evaluations:v0.1.0 .
+    docker build -t us-west1-docker.pkg.dev/$PROJECT_ID/my-herodotus/evaluations:v0.2.0 .
   automapSubstitutions: true
 images:
-- 'us-west1-docker.pkg.dev/$PROJECT_ID/my-herodotus/evaluations:v0.1.0'
+- 'us-west1-docker.pkg.dev/$PROJECT_ID/my-herodotus/evaluations:v0.2.0'

services/reddit-tool/.gcloudignore

@@ -0,0 +1,5 @@
+pytest.ini
+tool_test.pytest
+env/
+__pycache__
+.pytest_cache

services/reddit-tool/main.py

@@ -0,0 +1,59 @@
+import os
+import functions_framework
+
+import vertexai
+from vertexai.preview import reasoning_engines
+
+from tool import get_reddit_reviews
+
+LOCATION = "us-west1"
+MODEL = "gemini-1.5-pro"
+
+
+@functions_framework.http
+def get_agent_request(request):
+    """HTTP Cloud Function.
+    Args:
+        request (flask.Request): The request object.
+        <https://flask.palletsprojects.com/en/1.1.x/api/#incoming-request-data>
+    Returns:
+        The response text, or any set of values that can be turned into a
+        Response object using `make_response`
+        <https://flask.palletsprojects.com/en/1.1.x/api/#flask.make_response>.
+    """
+    query = ""
+    request_json = request.get_json(silent=True)
+
+    if request_json and "query" in request_json:
+        query = request_json["query"]
+
+    project_id = os.environ["PROJECT_ID"]
+    staging_bucket = os.environ["BUCKET"]
+
+    vertexai.init(project=project_id, location=LOCATION,
+                  staging_bucket=staging_bucket)
+
+    system_instruction = """
+You are a helpful AI travel assistant. The user wants to hear Reddit reviews
+about a specific location. You are going to use the get_reddit_reviews tool to
+get Reddit posts about the specific location that the user wants to know about.
+"""
+
+    agent = reasoning_engines.LangchainAgent(
+        system_instruction=system_instruction,
+        model=MODEL,
+        # Try to avoid "I can't help you" answers
+        model_kwargs={"temperature": 0.6},
+        tools=[
+            get_reddit_reviews,
+        ],
+    )
+
+    response = agent.query(
+        input=query
+    )
+    output = response["output"]
+
+    return {
+        "response": output
+    }

services/reddit-tool/requirements.txt

@@ -1,7 +1,7 @@
 praw
-google-cloud-aiplatform
 google-cloud-secret-manager
+google-cloud-aiplatform
 google-cloud-aiplatform[langchain,reasoningengine]
-cloudpickle==3.0.0
+pytest
 pydantic==2.7.4
-pytest
+functions-framework

services/reddit-tool/tool.py

@@ -17,14 +17,17 @@
 LOCATION = "us-central1"
 MODEL = "gemini-1.5-pro"
 
+
 def get_secrets() -> Mapping[str, str]:
     secret_name = f"projects/{PROJECT_ID}/secrets/reddit-api-key/versions/1"
     secret_client = secretmanager.SecretManagerServiceClient()
     secret = secret_client.access_secret_version(name=secret_name)
     reddit_key_json = json.loads(secret.payload.data)
     return reddit_key_json
 
-def get_posts(query: str, credentials: Mapping[str, str]) -> List[Mapping[str, str]]:
+
+def get_posts(query: str, credentials: Mapping[str, str]
+              ) -> List[Mapping[str, str]]:
     reddit = praw.Reddit(
         client_id=credentials["client_id"],
         client_secret=credentials["secret"],
@@ -43,6 +46,7 @@ def get_posts(query: str, credentials: Mapping[str, str]) -> List[Mapping[str, s
         })
     return reddit_messages
 
+
 def get_reddit_reviews(query: str) -> List[Mapping[str, str]]:
     """Gets a list of place reviews from Reddit.
 
@@ -56,21 +60,24 @@ def get_reddit_reviews(query: str) -> List[Mapping[str, str]]:
     messages = get_posts(query, credentials=reddit_key_json)
     return messages
 
+
 def deploy():
     project_id = os.environ["PROJECT_ID"]
     staging_bucket = os.environ["BUCKET"]
-    vertexai.init(project=project_id, location=LOCATION, staging_bucket=staging_bucket)
+    vertexai.init(project=project_id, location=LOCATION,
+                  staging_bucket=staging_bucket)
 
     system_instruction = """
-You are a helpful AI travel assistant. The user wants to hear Reddit reviews about a
-specific location. You are going to use the get_reddit_reviews tool to get Reddit posts
-about the specific location that the user wants to know about.
+You are a helpful AI travel assistant. The user wants to hear Reddit reviews
+about a specific location. You are going to use the get_reddit_reviews tool to
+get Reddit posts about the specific location that the user wants to know about.
 """
 
     agent = reasoning_engines.LangchainAgent(
         system_instruction=system_instruction,
         model=MODEL,
-        model_kwargs={"temperature": 0.6}, # Try to avoid "I can't help you" answers
+        # Try to avoid "I can't help you" answers
+        model_kwargs={"temperature": 0.6},
         tools=[
             get_reddit_reviews,
         ],
@@ -92,11 +99,12 @@ def deploy():
 
     # Test remote
     response = remote_agent.query(
-        input="""I want to take a trip to Crete. Where should I stay? What sites should I go see?"""
+        input="""I want to take a trip to Crete. Where should I stay? What
+    sites should I go see?"""
     )
     output = response["output"]
     print(output)
 
 
 if __name__ == "__main__":
-    deploy()
+    deploy()

services/reddit-tool/tool_test.py

@@ -10,16 +10,19 @@
 MODEL = "gemini-1.5-pro"
 REASONING_ENGINE_ID = "1823623597550206976"
 LOGGER = logging.getLogger()
-INPUT = """I want to take a trip to Crete. Where should I stay? I want to see ancient ruins. What are the best archaeological sites to see?"""
+INPUT = """I want to take a trip to Crete. Where should I stay? I want to see
+ancient ruins. What are the best archaeological sites to see?"""
+
 
 def test_create_agent_local():
     project_id = os.environ["PROJECT_ID"]
     staging_bucket = os.environ["BUCKET"]
-    vertexai.init(project=project_id, location=LOCATION, staging_bucket=staging_bucket)
+    vertexai.init(project=project_id, location=LOCATION,
+                  staging_bucket=staging_bucket)
     system_instruction = """
-You are a helpful AI travel assistant. The user wants to hear Reddit reviews about a
-specific location. You are going to use the get_reddit_reviews tool to get Reddit posts
-about the specific location that the user wants to know about.
+You are a helpful AI travel assistant. The user wants to hear Reddit reviews
+about a specific location. You are going to use the get_reddit_reviews tool to
+get Reddit posts about the specific location that the user wants to know about.
 """
     agent = reasoning_engines.LangchainAgent(
         model=MODEL,
@@ -41,9 +44,11 @@ def test_create_agent_local():
 def test_query_agent_remote():
     project_number = os.environ["PROJECT_NUMBER"]
     agent_name = f'projects/{project_number}/locations/us-central1/reasoningEngines/{REASONING_ENGINE_ID}'
-    reasoning_engine = vertexai.preview.reasoning_engines.ReasoningEngine(agent_name)
+    reasoning_engine = vertexai.preview.reasoning_engines.ReasoningEngine(
+        agent_name)
     response = reasoning_engine.query(
-        input="""I want to take a trip to Crete. Where should I stay? I want to see ancient ruins. What are the best archaeological sites to see?"""
+        input="""I want to take a trip to Crete. Where should I stay? I want
+        to see ancient ruins. What are the best archaeological sites to see?"""
     )
     output = response['output']