# Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond

**Authors:** Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, Jingren Zhou  
**arXiv:** [2308.12966](https://arxiv.org/abs/2308.12966)  
**Date:** 2023-08-24  
**Summary page:** https://www.summarizepaper.com/en/paper/2308.12966v3/

## Key points

- Authors introduce Qwen-VL series of large-scale vision-language models (LVLMs)
- Qwen-VL models designed to perceive and comprehend both texts and images
- Enhanced capabilities include visual receptor, input-output interface, 3-stage training pipeline, and multilingual multimodal cleaned corpus
- Advanced abilities such as grounding and text-reading through alignment of image-caption-box tuples
- Improved performance on generalist tasks like image captioning, question answering, and visual grounding in zero-shot and few-shot scenarios
- Qwen-VL-Chat demonstrates superiority over existing vision-language chatbots in real-world dialog benchmarks
- Codebase available on GitHub at https://github.com/QwenLM/Qwen-VL with demos and pre-trained models

## Summary

, , , , 

In their paper titled "Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond," authors Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou introduce the Qwen-VL series of large-scale vision-language models (LVLMs). These models are specifically designed to perceive and comprehend both texts and images. Building upon the foundation of the Qwen-LM model, they enhance its capabilities by incorporating a meticulously designed visual receptor, input-output interface, 3-stage training pipeline, and a multilingual multimodal cleaned corpus. The Qwen-VL models go beyond traditional image description and question-answering tasks by implementing advanced abilities such as grounding and text-reading through the alignment of image-caption-box tuples. This results in improved performance on generalist tasks across a wide range of visual-centric benchmarks including image captioning, question answering, and visual grounding in various settings such as zero-shot and few-shot scenarios. Additionally,<Organization> Qwen-VL-Chat </Organization> has demonstrated superiority over existing vision-language chatbots in real-world dialog benchmarks. The authors have made their codebase available on GitHub at https://github.com/QwenLM/Qwen-VL along with demos and pre-trained models. This work showcases the potential of LVLMs in bridging the gap between text understanding and image perception while achieving state-of-the-art performance across multiple vision-language tasks.