Remove push_to_index option from run-eval.yml workflow

[juanmichelini]

Summary

Remove the push_to_index option from the run-eval.yml workflow as it is being removed from downstream actions.

Fixes #1749

Changes

• Removed the push_to_index input parameter definition from workflow_dispatch inputs
• Removed the PUSH_TO_INDEX environment variable from the dispatch evaluation workflow step
• Removed the push_to_index argument from the jq command that builds the payload
• Removed the push_to_index field from the JSON payload sent to the evaluation workflow

Checklist

• If the PR is changing/adding functionality, are there tests to reflect this?
  • N/A - This is a configuration file change, no tests needed
• If there is an example, have you run the example to make sure that it works?
  • N/A - No examples affected
• If there are instructions on how to run the code, have you followed the instructions and made sure that it works?
  • N/A - No instructions affected
• If the feature is significant enough to require documentation, is there a PR open on the OpenHands/docs repository with the same branch name?
  • N/A - No documentation needed
• Is the github CI passing?
  • Pre-commit hooks passed locally

@juanmichelini can click here to continue refining the PR

---

Agent Server images for this PR

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant	Architectures	Base Image	Docs / Tags
java	amd64, arm64	eclipse-temurin:17-jdk	Link
python	amd64, arm64	nikolaik/python-nodejs:python3.12-nodejs22	Link
golang	amd64, arm64	golang:1.21-bookworm	Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:2741856-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-2741856-python \
  ghcr.io/openhands/agent-server:2741856-python

All tags pushed for this build

ghcr.io/openhands/agent-server:2741856-golang-amd64
ghcr.io/openhands/agent-server:2741856-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:2741856-golang-arm64
ghcr.io/openhands/agent-server:2741856-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:2741856-java-amd64
ghcr.io/openhands/agent-server:2741856-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:2741856-java-arm64
ghcr.io/openhands/agent-server:2741856-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:2741856-python-amd64
ghcr.io/openhands/agent-server:2741856-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-amd64
ghcr.io/openhands/agent-server:2741856-python-arm64
ghcr.io/openhands/agent-server:2741856-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-arm64
ghcr.io/openhands/agent-server:2741856-golang
ghcr.io/openhands/agent-server:2741856-java
ghcr.io/openhands/agent-server:2741856-python

About Multi-Architecture Support

• Each variant tag (e.g., 2741856-python) is a multi-arch manifest supporting both amd64 and arm64
• Docker automatically pulls the correct architecture for your platform
• Individual architecture tags (e.g., 2741856-python-amd64) are also available if needed

[all-hands-bot]

Code Review Summary

✅ Overall Assessment: The PR cleanly and completely removes all references to push_to_index from the workflow. The changes are syntactically correct.

What Was Verified

✅ Completeness: All four references removed:

• Input parameter definition (lines 73-77)
• Environment variable assignment (line 243)
• jq argument (line 261)
• JSON payload field (line 263)

✅ Syntax Validation:

• YAML syntax is valid
• jq command produces correct JSON output (tested locally)
• No trailing commas or structural issues

✅ No Orphaned References: Searched the entire codebase - no other files reference push_to_index

Minor Observations

🟡 Line Length: The jq command on line 258 is 450 characters long, which exceeds typical style guidelines. However, this is a pre-existing issue that was actually improved by this PR (reduced from ~478 characters). Consider splitting this into multiple lines in a future refactoring for better readability.

🟢 Downstream Coordination: The PR description states that push_to_index "is being removed from downstream actions". Please confirm that the downstream workflow in OpenHands/evaluation repo (specifically eval-job.yml) has been updated accordingly to prevent integration issues.

Recommendation

The changes are technically sound and ready to merge once downstream coordination is confirmed.

[openhands-ai]

Looks like there are a few issues preventing this PR from being merged!

• GitHub Actions are failing:
  • Run Eval (swebenchmultimodal) full claude-4.5-sonnet

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #1750 at branch `openhands/remove-push-to-index-option`

Feel free to include any additional details that might help me get this PR into a better state.

_{^{You can manage your notification settings}}