Hybrid Transformer and CNN Attention Network for Stereo Image Super-resolution
AI-generated Key Points
- Multi-stage strategies commonly used in image restoration tasks
- Transformer-based methods successful in single-image super-resolution tasks
- No significant advantages of transformers over CNN-based methods in stereo super-resolution tasks due to two main factors:
- Single-image super-resolution transformers cannot effectively utilize complementary stereo information
- Transformers rely on large amounts of training data lacking in common stereo-image super-resolution algorithms
- Authors propose a Hybrid Transformer and CNN Attention Network (HTCAN) for stereo image super-resolution
- HTCAN combines transformer-based network for single-image enhancement with CNN-based network for stereo information fusion
- Multi-patch training strategy and larger window sizes used to activate more input pixels for super resolution
- Other advanced techniques such as data augmentation, data ensemble, and model ensemble employed to reduce overfitting and data bias
- Proposed approach achieved a score of 23.90dB and emerged as the winner in Track 1 of the NTIRE 2023 Stereo Image Super Resolution Challenge
- Importance emphasized of utilizing information from both views in stereo image super resolution
- Feature extraction capability of each view and exchange of stereo information play crucial roles in determining final performance
- Transformers suitable for stereo image super resolution due to larger receptive fields and self attention mechanisms that effectively model long range dependencies
- Transformers have higher memory and computational costs compared to CNNs, which becomes challenging with high resolution images and large number of query tokens
- CNN-based models can afford more parallel exchange modules allowing for more thorough information exchange
Authors: Ming Cheng, Haoyu Ma, Qiufang Ma, Xiaopeng Sun, Weiqi Li, Zhenyu Zhang, Xuhan Sheng, Shijie Zhao, Junlin Li, Li Zhang
Abstract: Multi-stage strategies are frequently employed in image restoration tasks. While transformer-based methods have exhibited high efficiency in single-image super-resolution tasks, they have not yet shown significant advantages over CNN-based methods in stereo super-resolution tasks. This can be attributed to two key factors: first, current single-image super-resolution transformers are unable to leverage the complementary stereo information during the process; second, the performance of transformers is typically reliant on sufficient data, which is absent in common stereo-image super-resolution algorithms. To address these issues, we propose a Hybrid Transformer and CNN Attention Network (HTCAN), which utilizes a transformer-based network for single-image enhancement and a CNN-based network for stereo information fusion. Furthermore, we employ a multi-patch training strategy and larger window sizes to activate more input pixels for super-resolution. We also revisit other advanced techniques, such as data augmentation, data ensemble, and model ensemble to reduce overfitting and data bias. Finally, our approach achieved a score of 23.90dB and emerged as the winner in Track 1 of the NTIRE 2023 Stereo Image Super-Resolution Challenge.
Ask questions about this paper to our AI assistant
You can also chat with multiple papers at once here.
Assess the quality of the AI-generated content by voting
Score: 0
Why do we need votes?
Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.
The previous summary was created more than a year ago and can be re-run (if necessary) by clicking on the Run button below.
Similar papers summarized with our AI tools
Navigate through even more similar papers through a
tree representationLook for similar papers (in beta version)
By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.
Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.