# Partially fake it till you make it: mixing real and fake thermal images for improved object detection

**Authors:** Francesco Bongini, Lorenzo Berlincioni, Marco Bertini, Alberto Del Bimbo  
**arXiv:** [2106.13603](https://arxiv.org/abs/2106.13603)  
**Date:** 2021-06-25  
**Summary page:** https://www.summarizepaper.com/en/paper/2106.13603v1/

## Key points

- Francesco Bongini, Lorenzo Berlincioni, Marco Bertini, and Alberto Del Bimbo propose a novel approach for augmenting visual content domains with limited training datasets.
- The approach involves compositing synthetic 3D objects within real scenes to enhance object detection in thermal videos.
- Creating realistic synthetic scenes can be challenging due to the complexities of modeling thermal properties.
- The authors compare various augmentation strategies including reinforcement learning methods, injecting simulated data, and utilizing generative models.
- Their approach significantly improves object detection performance and achieves state-of-the-art results on the FLIR ADAS dataset.
- Multiple augmentation strategies are tested by introducing synthetic data into the training set categorized into different sets such as Syntha, Synthb, and Synthc.
- Ablation studies are conducted to evaluate the impact of these synthetic datasets on detector performance.
- Experiments involving generative models trained on specific subsets of synthetic data for inference tasks are explored.

## Summary

In their paper titled "Partially fake it till you make it: mixing real and fake thermal images for improved object detection," Francesco Bongini, Lorenzo Berlincioni, Marco Bertini, and Alberto Del Bimbo propose a novel approach for augmenting visual content domains with limited training datasets. The approach involves compositing synthetic 3D objects within real scenes to enhance object detection in thermal videos. This is particularly beneficial in scenarios where training datasets are scarce compared to visible spectrum datasets. Creating realistic synthetic scenes can be challenging due to the complexities of modeling thermal properties. The authors compare various augmentation strategies including state-of-the-art techniques obtained through reinforcement learning (RL) methods, injecting simulated data, and utilizing generative models. They conduct experiments to determine the effectiveness of combining their proposed augmentation method with these existing techniques. The results demonstrate that their approach significantly improves object detection performance. Their single-modality detector achieves state-of-the-art results on the FLIR ADAS dataset. Furthermore, the authors devise and test multiple augmentation strategies by combining different sources of data augmentation. They introduce synthetic data into the training set which is categorized into sets such as Syntha (pedestrians walking on a railroad scene), Synthb (cars and pedestrians over FLIR-ADAS scenes), and Synthc (cars and pedestrians on a railroad scene). Ablation studies are conducted to evaluate the impact of these synthetic datasets on detector performance. Additionally, the authors explore experiments involving generative models trained on specific subsets of synthetic data for inference tasks. They also test a generative model capable of translating RGB images to thermal images. Ablation studies are performed to analyze the effectiveness of different data augmentation strategies in improving detector performance. Overall, this study showcases the efficacy of combining synthetic data augmentation with existing techniques for enhancing object detection in thermal videos. The detailed experimentation and analysis presented in the paper contribute valuable insights to the field of computer vision research.