Skip to content

@METR METR

Change the repository type filter

All

    Repositories list

    25 repositories

    • vivaria

      Public
      Vivaria is METR's tool for running evaluations and conducting agent elicitation research.
      aielicitationai-evaluationevals
      TypeScript
      •
      MIT License
      •28•78•226•15•Updated Feb 18, 2025Feb 18, 2025

    • inspect_k8s_sandbox

      Public
      A Kubernetes sandbox environment for use with inspect_ai
      Python
      •
      MIT License
      •3•0•0•0•Updated Feb 14, 2025Feb 14, 2025

    • autonomy-evals-guide

      Public
      SCSS
      •
      MIT License
      •4•3•0•2•Updated Feb 14, 2025Feb 14, 2025

    • SWE-bench-fork

      Public
      [ICLR 2024] SWE-bench: Can Language Models Resolve Real-world Github Issues?
      Python
      •
      MIT License
      •415•0•0•0•Updated Feb 11, 2025Feb 11, 2025

    • KernelBenchFiltered

      Public
      Python
      •1•1•0•0•Updated Feb 11, 2025Feb 11, 2025

    • eval-analysis-public

      Public
      Public repository containing METR's DVC pipeline for eval data analysis
      Python
      •3•2•5•1•Updated Feb 10, 2025Feb 10, 2025

    • viv-task-dev

      Public
      Shell
      •1•0•7•0•Updated Feb 6, 2025Feb 6, 2025

    • inspect_ai

      Public
      Inspect: A framework for large language model evaluations
      Python
      •
      MIT License
      •188•0•0•0•Updated Feb 4, 2025Feb 4, 2025

    • task-standard

      Public
      METR Task Standard
      TypeScript
      •
      MIT License
      •32•142•6•3•Updated Feb 3, 2025Feb 3, 2025

    • ai-rd-tasks

      Public
      Python
      •
      Other
      •6•61•3•1•Updated Jan 30, 2025Jan 30, 2025

    • agent-fork-of-eval-analysis-public

      Public archive
      For of eval-analysis-public repo intended for agent PRs
      Python
      •3•0•0•0•Updated Jan 28, 2025Jan 28, 2025

    • llm-foundry

      Public
      LLM training code for Databricks foundation models
      Python
      •
      Apache License 2.0
      •544•0•0•0•Updated Jan 28, 2025Jan 28, 2025

    • metr-task-boilerplate

      Public
      A Cookiecutter template for developing tasks according to the METR Task Standard
      evaluationssafetyllm
      TypeScript
      •0•1•0•0•Updated Jan 22, 2025Jan 22, 2025

    • task-protected-scoring

      Public
      Python
      •1•0•3•2•Updated Jan 21, 2025Jan 21, 2025

    • task-artifacts

      Public
      Python
      •0•0•0•0•Updated Jan 21, 2025Jan 21, 2025

    • public-tasks

      Public
      TeX
      •
      Other
      •7•83•0•1•Updated Jan 9, 2025Jan 9, 2025

    • worktest-sw-eng-deps

      Public
      Dockerfile
      •0•0•0•0•Updated Dec 29, 2024Dec 29, 2024

    • task-assets

      Public
      Python
      •0•0•1•0•Updated Dec 28, 2024Dec 28, 2024

    • .github

      Public
      0•0•0•0•Updated Nov 24, 2024Nov 24, 2024

    • nanoGPT

      Public
      The simplest, fastest repository for training/finetuning medium-sized GPTs.
      Python
      •
      MIT License
      •6.4k•0•0•0•Updated Nov 22, 2024Nov 22, 2024

    • task-legacy-verifier

      Public
      Python
      •0•0•0•0•Updated Nov 8, 2024Nov 8, 2024

    • task-aux-vm-helpers

      Public
      Python
      •1•0•0•0•Updated Nov 2, 2024Nov 2, 2024

    • pyhooks

      Public archive
      A library that METR agents use to communicate with Vivaria.
      Python
      •1•0•1•0•Updated Sep 22, 2024Sep 22, 2024

    • vivaria-mentat

      Public archive
      Vivaria is METR's tool for running evaluations and conducting agent elicitation research.
      TypeScript
      •
      MIT License
      •28•0•1•1•Updated Sep 19, 2024Sep 19, 2024

    • task-template

      Public template
      TypeScript
      •6•9•2•3•Updated Aug 6, 2024Aug 6, 2024