Skip to content

Commit

Permalink
update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
rayguan97 committed Oct 31, 2023
1 parent 6b78700 commit d0d6f7f
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,15 @@ To keep evaluation simple, we only provide the question in form of yes/no questi

### Evaluation

1. git the repo
1. Clone the repo.
```
git clone https://github.com/tianyi-lab/HallusionBench.git
cd ./HallusionBench
```

2. Download the images [hallusion_bench.zip](https://drive.google.com/file/d/1sAXmVg3I3A6gjb8JnA6HWjSv1ntrSBki/view?usp=drive_link) and unzip the folder in the same directory.

3. The Questions and image locations are saved in `./HallusionBench.json`. The data sample are as follows:
3. The questions and image locations are saved in `./HallusionBench.json`. The data sample are as follows:
```
{'category': 'VD', 'subcategory': 'illusion', 'visual_input': '1', 'set_id': '0', 'figure_id': '0', 'sample_note': 'circle', 'question_id': '0', 'question': 'Is the right orange circle the same size as the left orange circle?', 'gt_answer_details': 'The right orange circle is the same size as the left orange circle.', 'gt_answer': '1', 'filename': './hallusion_bench/VD/illusion/0_0.png'}
```
Expand All @@ -56,7 +56,7 @@ The key `visual_input`means whether the question needs visual input like images.
python evaluation.py
```

We provide the API key in the script for convenience. You can also use your own API key for GPT4 evaluation by editing the code [here](./evaluation.py#L9)
We provide the API key in the script for convenience. You can also use your own API key for GPT4 evaluation by editing the code [here](./evaluation.py#L9).



Expand Down

0 comments on commit d0d6f7f

Please sign in to comment.