# PFT-SSR: Parallax Fusion Transformer for Stereo Image Super-Resolution

**Authors:** Hansheng Guo, Juncheng Li, Guangwei Gao, Zhi Li, Tieyong Zeng  
**arXiv:** [2303.13807](https://arxiv.org/abs/2303.13807)  
**Date:** 2023-03-24  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.13807v1/

## Key points

- Stereo image super-resolution is a technique that enhances the performance of image super-resolution by utilizing additional information provided by binocular systems.
- Previous methods did not fully utilize cross-view and intra-view information.
- A Parallax Fusion Transformer (PFT) module was proposed to address this issue, consisting of Cross-view Fusion Transformer (CVFT) and Intra-view Refinement Transformer (IVRT).
- CVFT merges different parallaxes and extracts features from both left and right images while IVRT refines the intra-view features by removing noise and enhancing texture details.
- PFT was combined with Swin Transformer as the backbone for feature extraction and SR reconstruction to form a pure Transformer architecture called PFT-SSR.
- Extensive experiments were conducted to evaluate the effectiveness of PFT-SSR compared to other state-of-the-art methods such as EDSR, RCAN, StereoSR, PASSRnet, iPASSR, and SSRDE-FNett.
- The results showed that PFT–SSR outperformed most SOTA methods on various datasets, especially on Flickr102.
- An ablation study was conducted on the choice of cross–view interaction technology to demonstrate the strong stereo image fusion ability of PFT which improved model performance more effectively than Swin Transformer Blocks or biPAM.
- In conclusion, PFT–SSR is a promising approach for stereo image super–resolution that fully utilizes cross–view and intra–view information through its innovative Parallax Fusion Transformer module.
- Experiments demonstrated its superiority over other state–of–the–art methods in terms of accuracy in reconstructing edges and texture details.
- The source code for PFT–SSR is available on GitHub.

## Summary

Stereo image super-resolution is a technique that utilizes the additional information provided by binocular systems to enhance the performance of image super-resolution. Previous methods have not fully utilized the cross-view and intra-view information. To address this issue, researchers proposed a Parallax Fusion Transformer (PFT) module consisting of two components: Cross-view Fusion Transformer (CVFT) and Intra-view Refinement Transformer (IVRT). CVFT merges different parallaxes and extracts features from both left and right images while IVRT refines the intra-view features by removing noise and enhancing texture details. The team then combined PFT with Swin Transformer as the backbone for feature extraction and SR reconstruction to form a pure Transformer architecture called PFT-SSR. Extensive experiments and ablation studies were conducted to evaluate the effectiveness of PFT-SSR compared to other state-of-the art methods such as EDSR, RCAN, StereoSR, PASSRnet, iPASSR, and SSRDE–FNett. The results showed that PFT–SSR outperformed most SOTA methods on various datasets, especially on Flickr102. Additionally, an ablation study was conducted on the choice of cross–view interaction technology to demonstrate the strong stereo image fusion ability of PFT which improved model performance more effectively than Swin Transformer Blocks or biPAM. In conclusion, PFT–SSR is a promising approach for stereo image super–resolution that fully utilizes cross–view and intra–view information through its innovative Parallax Fusion Transformer module. Experiments demonstrated its superiority over other state–of–the–art methods in terms of accuracy in reconstructing edges and texture details. The source code for PFT–SSR is available on GitHub.