fix: initialize default metadata with all required fields

[xingyaoww]

This PR fixes an issue where the evaluation script would error out if metadata.json doesn't exist. Now it will initialize a default EvalMetadata with all required fields when the file is missing.

Changes:

• Added a fallback to create default metadata when metadata.json doesn't exist
• Initializes EvalMetadata with all required fields:
  • agent_class: "dummy_agent" (placeholder)
  • llm_config: LLMConfig with "dummy_model"
  • max_iterations: 1
  • eval_output_dir: input file directory
  • start_time: current time
  • git_commit: current commit hash
  • dataset: from args.dataset
• Maintains type safety by ensuring metadata is always an EvalMetadata instance

---

To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:543d0ba-nikolaik   --name openhands-app-543d0ba   docker.all-hands.dev/all-hands-ai/openhands:543d0ba

[csmith49]

LGTM.

Is it worth moving the defaults to the fallbacks in EvalMetadata so they're shared across all benchmarks? Looks pretty specific to SWE-bench.

[xingyaoww]

Is it worth moving the defaults to the fallbacks in EvalMetadata so they're shared across all benchmarks?

I think most other benchmarks won't have "eval_infer.py" which is pretty special/specific to SWE-Bench (e.g., only doing patch evaluation, but not inference), so I think having this being special case for SWE-Bench is probably ok?