From 9ae53932daa44210e490159dade604e1087c2e23 Mon Sep 17 00:00:00 2001 From: Wenhao Chai Date: Sat, 26 Oct 2024 08:02:59 -0700 Subject: [PATCH 1/2] Update README.md --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 00626e08..a874d97b 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,8 @@ --- ## Annoucement -- [2024-09] 🎉🎉 We welcome the new tasks [MMSearch](https://mmsearch.github.io/) and [MME-RealWorld](https://mme-realworld.github.io/) for inference acceleration +- [2024-10] 🎉🎉 We welcome the new tasks [VDC](https://rese1f.github.io/aurora-web/) for video detailed captioning and [MovieChat-1K](https://rese1f.github.io/MovieChat/) for long-form video understanding. We also welcome the two video-lmms baselines [AuroraCap](https://github.com/rese1f/aurora) and [MovieChat](https://github.com/rese1f/MovieChat). +- [2024-09] 🎉🎉 We welcome the new tasks [MMSearch](https://mmsearch.github.io/) and [MME-RealWorld](https://mme-realworld.github.io/) for inference acceleration - [2024-09] ⚙️️⚙️️️️ We upgrade `lmms-eval` to `0.2.3` with more tasks and features. We support a compact set of language tasks evaluations (code credit to [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness)), and we remove the registration logic at start (for all models and tasks) to reduce the overhead. Now `lmms-eval` only launches necessary tasks/models. Please check the [release notes](https://github.com/EvolvingLMMs-Lab/lmms-eval/releases/tag/v0.2.3) for more details. - [2024-08] 🎉🎉 We welcome the new model [LLaVA-OneVision](https://huggingface.co/papers/2408.03326), [Mantis](https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/162), new tasks [MVBench](https://huggingface.co/datasets/OpenGVLab/MVBench), [LongVideoBench](https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/117), [MMStar](https://github.com/EvolvingLMMs-Lab/lmms-eval/pull/158). We provide new feature of SGlang Runtime API for llava-onevision model, please refer the [doc](https://github.com/EvolvingLMMs-Lab/lmms-eval/blob/main/docs/commands.md) for inference acceleration - [2024-07] 🎉🎉 We have released the [technical report](https://arxiv.org/abs/2407.12772) and [LiveBench](https://huggingface.co/spaces/lmms-lab/LiveBench)! From d890e10b9d5ba188ce1ec4388a432aa65a8c839f Mon Sep 17 00:00:00 2001 From: Wenhao Chai Date: Sat, 26 Oct 2024 08:08:24 -0700 Subject: [PATCH 2/2] add comment for VDC and MovieChat --- docs/current_tasks.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/current_tasks.md b/docs/current_tasks.md index 2cdc3bc8..2c1b4333 100644 --- a/docs/current_tasks.md +++ b/docs/current_tasks.md @@ -216,6 +216,9 @@ - egoschema_subset_mcppl - egoschema_subset - [LongVideoBench](https://github.com/longvideobench/LongVideoBench) +- [MovieChat](https://github.com/rese1f/MovieChat) (moviechat) + - Global Mode for entire video (moviechat_global) + - Breakpoint Mode for specific moments (moviechat_breakpoint) - [MLVU](https://github.com/JUNJIE99/MLVU) (mlvu) - [MMT-Bench](https://mmt-bench.github.io/) (mmt) - MMT Validation (mmt_val) @@ -292,11 +295,8 @@ - [YouCook2](http://youcook2.eecs.umich.edu/) (youcook2_val) -- [MovieChat](https://github.com/rese1f/MovieChat) (moviechat) - - MovieChat Global Model (moviechat_global) - - MovieChat Breakpoint Model (moviechat_breakpoint) - - [VDC](https://github.com/rese1f/aurora) (vdc) + - VDC Detailed Caption (detailed_test) - VDC Camera Caption (camera_test) - VDC Short Caption (short_test) - VDC Background Caption (background_test)