# Sequential Modeling Enables Scalable Learning for Large Vision Models

**Authors:** Yutong Bai, Xinyang Geng, Karttikeya Mangalam, Amir Bar, Alan Yuille, Trevor Darrell, Jitendra Malik, Alexei A Efros  
**arXiv:** [2312.00785](https://arxiv.org/abs/2312.00785)  
**Date:** 2023-12-01  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2312.00785v1/

## Key points

- Authors introduce a sequential modeling method for Large Vision Models (LVM)
- Method represents raw images and visual data without additional meta-knowledge
- Model trained to predict next token in sequences of 420 billion tokens
- Demonstrated scalability through training across various model architectures and data diversity
- Approach effective in tackling vision tasks by designing appropriate visual prompts
- Research showcases potential for learning LVMs without linguistic data
- Sequential modeling enhances scalability and performance in vision applications

## Summary

In their paper titled "Sequential Modeling Enables Scalable Learning for Large Vision Models," authors Yutong Bai, Xinyang Geng, Karttikeya Mangalam, Amir Bar, Alan Yuille, Trevor Darrell, Jitendra Malik, and Alexei A Efros introduce a groundbreaking approach that revolutionizes the learning process for Large Vision Models (LVM). This sequential modeling method represents raw images and other visual data sources without requiring any additional meta-knowledge beyond the pixel level. By converting this diverse range of visual data into sequences comprising a staggering 420 billion tokens, the model is trained to predict the next token in order to minimize cross-entropy loss. The authors demonstrate the remarkable scalability of their models through extensive training across various scales of model architecture and data diversity. They provide empirical evidence supporting the effectiveness of their approach in tackling numerous vision tasks by designing appropriate visual prompts during testing. This research not only showcases the potential for learning LVMs without linguistic data but also highlights the power of sequential modeling in enhancing scalability and performance in vision-related applications. The authors' work opens up new possibilities for advancing computer vision technology and underscores the importance of innovative approaches in pushing the boundaries of machine learning.