# E.T. Bench: Towards Open-Ended Event-Level Video-Language Understanding

**Authors:** Ye Liu, Zongyang Ma, Zhongang Qi, Yang Wu, Ying Shan, Chang Wen Chen  
**arXiv:** [2409.18111](https://arxiv.org/abs/2409.18111)  
**Date:** 2024-09-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2409.18111v1/

## Key points

- Authors introduced E.T. Bench as a comprehensive benchmark for open-ended event-level video understanding
- Existing benchmarks lack fine-grained event-level assessment and task diversity
- E.T. Bench includes 7.3K samples across 12 tasks within 8 domains, totaling 7K videos with a combined length of 251.4 hours
- The benchmark is categorized within a 3-level task taxonomy for thorough evaluations of models' capabilities in different scenarios
- State-of-the-art models struggle with fine-grained tasks such as grounding events-of-interest within videos due to limitations like short video context length, improper time representations, and insufficient multi-event training data
- Authors propose a strong baseline model called [model name] along with an instruction-tuning dataset named [dataset name] tailored for fine-grained event-level understanding
- Their solution demonstrates superior performance compared to existing models across multiple scenarios

## Summary

In their paper titled "E.T. Bench: Towards Open-Ended Event-Level Video-Language Understanding," authors Ye Liu, Zongyang Ma, Zhongang Qi, Yang Wu, Ying Shan, and Chang Wen Chen introduce E.T. Bench - a comprehensive benchmark designed to evaluate open-ended event-level video understanding. Recent advancements in Video Large Language Models (Video-LLMs) have shown promise in general video comprehension tasks. However, existing benchmarks primarily focus on video-level question-answering and lack fine-grained event-level assessment and task diversity. <br/>
<br/>
 have been making significant strides in the field of video comprehension technologies. In order to further advance this area of research, the authors identified a gap in current benchmarks and developed , which offers a large-scale and high-quality benchmark that includes 7.3K samples across 12 tasks within 8 domains - totaling 7K videos with a combined length of 251.4 hours.<br/>
<br/>
The benchmark is categorized within a 3-level task taxonomy to provide thorough evaluations of models' capabilities in different scenarios. The authors conducted extensive evaluations of 8 Image-LLMs and 12 Video-LLMs on the  and found that state-of-the-art models struggle with fine-grained tasks such as grounding events-of-interest within videos due to limitations such as short video context length, improper time representations, and insufficient multi-event training data.<br/>
<br/>
To address these challenges, the authors propose a strong baseline model called  along with an instruction-tuning dataset named  164K tailored for fine-grained event-level understanding. Their solution demonstrates superior performance across multiple scenarios compared to existing models.<br/>
<br/>
Overall, the introduction of  provides researchers and practitioners with a valuable tool for evaluating the capabilities of  in open-ended event-level video understanding tasks, highlighting the importance of fine-grained assessments in advancing the field of video comprehension technologies.