Skip to content

Commit

Permalink
remove mlebench
Browse files Browse the repository at this point in the history
  • Loading branch information
cyzus committed Oct 30, 2024
1 parent 261a39d commit 776ee44
Showing 1 changed file with 1 addition and 31 deletions.
32 changes: 1 addition & 31 deletions metagpt/ext/sela/runner/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,34 +165,4 @@ python run_experiment.py --exp_mode base --task titanic --num_experiments 10
To run additional baselines:

- Each baseline must produce `dev_predictions.csv` and `test_predictions.csv` with a `target` column.
- Use the `evaluate_score` function for evaluation.

---

## MLE-Bench

**Note:** MLE-Bench requires Python 3.11 or higher.

#### Setup

Clone the repository and install:

```bash
git clone https://github.com/openai/mle-bench.git
cd mle-bench
pip install -e .
```

Prepare the data:

```bash
mlebench prepare -c <competition-id> --data-dir <dataset-dir-save-path>
```

#### Run the MLE-Bench Experiment

Run the following command to execute the experiment:

```bash
python run_experiment.py --exp_mode mcts --custom_dataset_dir <dataset-dir-save-path/prepared/public> --rollouts 10 --from_scratch --role_timeout 3600
```
- Use the `evaluate_score` function for evaluation.

0 comments on commit 776ee44

Please sign in to comment.