Enable Laminar trajectory uploads in PR review workflow

[neubig]

Summary

This PR enables Laminar observability in the PR review workflow, allowing trajectories to be uploaded for monitoring and debugging.

Changes

• Added LMNR_PROJECT_API_KEY environment variable to the "Run PR review" step
• Mapped it to the existing LAMINAR_ACTIONS_API_KEY secret

How it works

The software-agent-sdk's Laminar integration looks for the LMNR_PROJECT_API_KEY environment variable to initialize observability. When set, the SDK will:

• Initialize Laminar via maybe_init_laminar()
• Append the LaminarLiteLLMCallback to litellm callbacks
• Automatically upload traces/trajectories to the Laminar dashboard

Requirements

The LAMINAR_ACTIONS_API_KEY secret must be configured in the repository settings for trajectories to be uploaded.

@neubig can click here to continue refining the PR

---

Agent Server images for this PR

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant	Architectures	Base Image	Docs / Tags
java	amd64, arm64	eclipse-temurin:17-jdk	Link
python	amd64, arm64	nikolaik/python-nodejs:python3.12-nodejs22	Link
golang	amd64, arm64	golang:1.21-bookworm	Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:7c09349-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-7c09349-python \
  ghcr.io/openhands/agent-server:7c09349-python

All tags pushed for this build

ghcr.io/openhands/agent-server:7c09349-golang-amd64
ghcr.io/openhands/agent-server:7c09349-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:7c09349-golang-arm64
ghcr.io/openhands/agent-server:7c09349-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:7c09349-java-amd64
ghcr.io/openhands/agent-server:7c09349-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:7c09349-java-arm64
ghcr.io/openhands/agent-server:7c09349-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:7c09349-python-amd64
ghcr.io/openhands/agent-server:7c09349-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-amd64
ghcr.io/openhands/agent-server:7c09349-python-arm64
ghcr.io/openhands/agent-server:7c09349-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-arm64
ghcr.io/openhands/agent-server:7c09349-golang
ghcr.io/openhands/agent-server:7c09349-java
ghcr.io/openhands/agent-server:7c09349-python

About Multi-Architecture Support

• Each variant tag (e.g., 7c09349-python) is a multi-arch manifest supporting both amd64 and arm64
• Docker automatically pulls the correct architecture for your platform
• Individual architecture tags (e.g., 7c09349-python-amd64) are also available if needed

[all-hands-bot]

Clean and straightforward change. The addition is minimal, well-commented, and follows the existing pattern. One minor suggestion about documentation.

[xingyaoww]

@OpenHands please merge from main and resolve conflicts

[openhands-ai]

I'm on it! xingyaoww can track my progress at all-hands.dev

[openhands-ai]

Successfully merged main into the enable-laminar-trajectory-uploads branch and resolved conflicts. The changes have been pushed to the remote.

Summary of Changes

The conflict in .github/workflows/pr-review-by-openhands.yml was resolved by:

1. Keeping the simplified action-based approach from main - The workflow now uses the reusable pr-review action instead of inline steps

2. Adding Laminar support to the pr-review action (.github/actions/pr-review/action.yml):

   • Added new optional input lmnr-project-api-key for Laminar trajectory uploads
   • Added LMNR_PROJECT_API_KEY environment variable in the "Run PR review" step

3. Enabling Laminar in the workflow (.github/workflows/pr-review-by-openhands.yml):

   • Added lmnr-project-api-key: ${{ secrets.LAMINAR_ACTIONS_API_KEY }} to pass the secret to the action

The PR now properly enables Laminar observability while using the updated action-based workflow structure from main.

View full conversation

[neubig]

@OpenHands check the changes made to the PR review process here: #1927

We'll want to update this PR to be compatible with those.

[openhands-ai]

I'm on it! neubig can track my progress at all-hands.dev

[openhands-ai]

I've reviewed PR #1927 and updated this PR to be fully compatible with the changes made to the PR review process.

Summary

PR #1927 refactored the PR review workflow to use a reusable composite GitHub Action. This branch already had the Laminar support properly integrated into:

• .github/actions/pr-review/action.yml - Added lmnr-project-api-key input and LMNR_PROJECT_API_KEY environment variable
• .github/workflows/pr-review-by-openhands.yml - Uses the new input with LAMINAR_ACTIONS_API_KEY secret

New Change

I added documentation for the lmnr-project-api-key input to the README:

• examples/03_github_workflows/02_pr_review/README.md - Added the lmnr-project-api-key input to the "Action Inputs" table

Checklist

• ✅ Reviewed PR refactor: Use composite GitHub Action for PR review workflow #1927 changes (composite action refactor)
• ✅ Verified this PR is compatible with the new composite action structure
• ✅ Updated README documentation to include the new lmnr-project-api-key input
• ✅ Changes are concise and focused only on Laminar support
• ✅ Pushed changes to the remote branch

The PR is now ready for review at #1947

View full conversation

[openhands-ai]

Looks like there are a few issues preventing this PR from being merged!

• GitHub Actions are failing:
  • Review Thread Gate

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #1947 at branch `enable-laminar-trajectory-uploads`

Feel free to include any additional details that might help me get this PR into a better state.

_{^{You can manage your notification settings}}

[xingyaoww]

@OpenHands pls merge from main and resolve merge conflicts

[openhands-ai]

I'm on it! xingyaoww can track my progress at all-hands.dev

[openhands-ai]

Successfully merged main into the enable-laminar-trajectory-uploads branch and resolved all merge conflicts. The changes have been pushed to the remote.

Summary of Changes

The merge conflicts in two files were resolved:

1. .github/actions/pr-review/action.yml:

   • Kept the standardized input name lmnr-api-key (from main) instead of lmnr-project-api-key
   • The input maps to the LMNR_PROJECT_API_KEY environment variable for the agent script
   • Removed unnecessary conditional override logic that was added in a previous commit

2. examples/03_github_workflows/02_pr_review/README.md:

   • Updated the Action Inputs table to use lmnr-api-key
   • Included the new "Review Evaluation (Observability)" documentation section from main

The branch now includes all the latest changes from main, including:

• LLMProfileStore for persisted LLM configurations
• PS1 metadata corruption fix
• PyInstaller delegate tool templates fix
• Planning agent clarifying questions enhancement
• Delayed evaluation for PR review with Laminar signals

View full conversation

[github-actions]

Coverage Report •

File	Stmts	Miss	Cover	Missing
openhands-sdk/openhands/sdk
__init__.py	20	2	90%	64–65
openhands-sdk/openhands/sdk/llm
llm.py	455	86	81%	400, 421–422, 458, 635, 736, 738–739, 767, 817, 828–830, 834–838, 846–848, 858–860, 863–864, 868, 870–871, 873, 896–901, 1024, 1029–1030, 1215–1216, 1225, 1238, 1240–1245, 1247–1264, 1267–1271, 1273–1274, 1280–1289, 1340, 1342
llm_profile_store.py	63	2	96%	51–52
openhands-sdk/openhands/sdk/llm/exceptions
types.py	59	2	96%	104, 109
openhands-tools/openhands/tools/terminal
metadata.py	50	5	90%	74–75, 96–97, 101
TOTAL	17664	5445	69%