Skip to content

Commit

Permalink
update acl 2024
Browse files Browse the repository at this point in the history
  • Loading branch information
khuangaf committed Jun 5, 2024
1 parent 9fa6720 commit fc7c564
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 12 deletions.
19 changes: 11 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Do LVLMs Understand Charts? Analyzing and Correcting Factual Errors in Chart Captioning
# [ACL 2024 Findings] Do LVLMs Understand Charts? Analyzing and Correcting Factual Errors in Chart Captioning

<div align="center">
<a href="https://khuangaf.github.io/">Kung-Hsiang Huang</a>†, Mingyang Zhou*, Hou Pong Chan‡,
Expand Down Expand Up @@ -91,7 +91,7 @@ Results are shown in the below figure and table. We found that all captioning mo
- [ ] Evaluation scripts


## The CHOCOLATE Benchmark
## The CHOCOLATE Benchmark

We release the data for the CHOCOLATE benchmark at `data/chocolate.json`. CHOCOLATE is also available on [HuggingFace](https://huggingface.co/datasets/khhuang/CHOCOLATE)🤗.

Expand Down Expand Up @@ -155,7 +155,7 @@ The meta-evaluation scripts can be found in [ChartVE Meta-evaluation.ipynb](http

The proposed C2TFEC framework consists of two components: chart-to-table conversion and table-based error rectification.

### Chart-To-Table
### Chart-To-Table

The Chart-To-Table model ([khhuang/chart-to-table](https://huggingface.co/khhuang/chart-to-table)) is trained to convert a chart into a structured table. The generated tables use &&& to delimit rows and | to delimit columns. The underlying architecture of this model is UniChart. Below, we provide an example of how to use our Chart-To-Table model.

Expand Down Expand Up @@ -208,7 +208,7 @@ extracted_table = sequence.split("<s_answer>")[1].strip()

## Citation
```bibtex
@misc{huang-etal-2023-do,
@inproceedings{huang-etal-2024-lvlms,
title = "Do LVLMs Understand Charts? Analyzing and Correcting Factual Errors in Chart Captioning",
author = "Huang, Kung-Hsiang and
Zhou, Mingyang and
Expand All @@ -218,10 +218,13 @@ extracted_table = sequence.split("<s_answer>")[1].strip()
Zhang, Lingyu and
Chang, Shih-Fu and
Ji, Heng",
year={2023},
eprint={2312.10160},
archivePrefix={arXiv},
primaryClass={cs.CL}
booktitle = "Findings of the Association for Computational Linguistics: ACL 2024",
month = aug,
year = "2024",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2023.findings-acl.85",
doi = "10.18653/v1/2023.findings-acl.85",
pages = "1314--1326",
}
```

8 changes: 4 additions & 4 deletions docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -367,7 +367,7 @@ <h2 class="title is-3" id="leaderboard">Leaderboard on Factual Inconsistency Det
<td><a href="https://bard.google.com/" class="ext-link" style="font-size: 16px;">Link</a></td>
<td>-0.014</td>
<td>0.105</td>
<td><b class="best-score-text">0.291</b></td>
<td><b>0.291</b></td>
</tr>

<tr>
Expand All @@ -386,13 +386,13 @@ <h2 class="title is-3" id="leaderboard">Leaderboard on Factual Inconsistency Det
<td><b>Large Vision-language Model</b></td>
<td><a href="https://openai.com/research/gpt-4v-system-card" class="ext-link" style="font-size: 16px;">Link</a></td>
<td>0.157</td>
<td>0.205</b></td>
<td><b>0.205</b></td>
<td>0.215</td>
</tr>

<tr>
<td></td>
<td><b>GPT-4O</b></td>
<td><b>GPT-4o</b></td>
<td><b>Large Vision-language Model</b></td>
<td><a href="https://openai.com/index/hello-gpt-4o/" class="ext-link" style="font-size: 16px;">Link</a></td>
<td><b class="best-score-text">0.250</td>
Expand All @@ -416,7 +416,7 @@ <h2 class="title is-3" id="leaderboard">Leaderboard on Factual Inconsistency Det
<td><b class="best-score-text"><span class="mathvista">ChartVE</span></b></td>
<td><b>Small Vision-language Model</b></td>
<td><a href="https://aclanthology.org/2023.findings-acl.660/" class="ext-link" style="font-size: 16px;">Link</a></td>
<td><b class="best-score-text">0.178</b></td>
<td><b>0.178</b></td>
<td>0.091</td>
<td>0.215</td>
</tr>
Expand Down

0 comments on commit fc7c564

Please sign in to comment.