# Street TryOn: Learning In-the-Wild Virtual Try-On from Unpaired Person Images

**Authors:** Aiyu Cui, Jay Mahajan, Viraj Shah, Preeti Gomathinayagam, Chang Liu, Svetlana Lazebnik  
**arXiv:** [2311.16094](https://arxiv.org/abs/2311.16094)  
**Date:** 2023-11-27  
**Summary page:** https://www.summarizepaper.com/en/paper/2311.16094v3/

## Key points

- Virtual try-on research has primarily focused on showcasing garments on studio models for the fashion industry in a cost-effective manner.
- There is a growing recognition of the need to extend virtual try-on technology to enable customers to visualize clothing items on themselves using their own everyday photos (in-the-wild try-on).
- Existing methods struggle with in-the-wild scenarios due to the reliance on paired data, which is more readily available for studio settings compared to diverse real-world scenes.
- A team of researchers led by Aiyu Cui, Jay Mahajan, Viraj Shah, Preeti Gomathinayagam, Chang Liu, and Svetlana Lazebnik introduced a groundbreaking approach:
- Established the StreetTryOn benchmark to support in-the-wild virtual try-on applications.
- Proposed an innovative method that learns virtual try-on directly from unpaired person images taken in natural settings without paired data.
- Utilized DensePose warping correction and diffusion-based conditional inpainting techniques to address challenges posed by in-the-wild scenarios.
- Through extensive experiments, the team demonstrated competitive performance for traditional studio try-on tasks and state-of-the-art results for street try-on and cross-domain try-on tasks.

## Summary

In the realm of virtual try-on research, the primary focus has been on catering to the fashion industry's needs by creating images that showcase garments on studio models in a cost-effective manner. However, there is a growing recognition that virtual try-on technology should extend beyond this narrow scope to enable customers to visualize clothing items on themselves using their own everyday photos, a concept known as in-the-wild try-on. The challenge lies in the fact that existing methods, which excel in generating realistic results for studio settings, struggle when applied to in-the-wild scenarios due to their reliance on paired data (garment images matched with images of individuals wearing those garments) for training. While obtaining such paired data is relatively straightforward for studio environments through shopping websites, it proves to be a significant hurdle in capturing diverse and uncontrolled real-world scenes. To address this gap, a team of researchers led by Aiyu Cui, Jay Mahajan, Viraj Shah, Preeti Gomathinayagam, Chang Liu, and Svetlana Lazebnik introduces a groundbreaking approach. They first establish the StreetTryOn benchmark aimed at supporting in-the-wild virtual try-on applications. Subsequently, they propose an innovative method that learns virtual try-on directly from a collection of unpaired person images taken in natural settings without the need for paired data. This novel technique tackles the unique challenges posed by in-the-wild scenarios by employing a DensePose warping correction method coupled with diffusion-based conditional inpainting. Through extensive experiments, the team demonstrates not only competitive performance for traditional studio try-on tasks but also state-of-the-art results for street try-on and cross-domain try-on tasks. In essence, this research represents a significant step forward in expanding the capabilities of virtual try-on technology beyond controlled studio environments towards more versatile and practical applications that cater to real-world scenarios where customers can seamlessly visualize clothing items on themselves using their own photographs.