# Trimmed Match Design for Randomized Paired Geo Experiments

**Authors:** Aiyou Chen, Marco Longfils, Nicolas Remy  
**arXiv:** [2105.07060](https://arxiv.org/abs/2105.07060)  
**Date:** 2021-05-14  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2105.07060v2/

## Key points

- The paper addresses the problem of measuring incremental Return On Ad Spend (iROAS) in the online advertising industry.
- The authors propose a robust nonparametric method called Trimmed Match Design (TMD) for designing reliable and cost-effective geo experiments.
- Geo experiments involve running randomized experiments in non-overlapping ad-targetable geographical areas.
- The TMD method addresses issues such as a small number of geos, heavy-tailed response metrics across geos due to geo heterogeneity, and varying response metrics over time.
- The proposed TMD method is described in detail along with its application in simulation studies and real case studies.
- Partial implementation code for TMD is available on GitHub [Chen et al., 2020].
- Previous work by Vaver & Koehler (2011), who proposed a two-stage regression model called geo-based regression (GBR) for analyzing randomized geo experiments, is mentioned.
- The paper highlights some open problems for future research in this area.
- The effectiveness of TMD as a reliable and cost-effective solution for measuring iROAS through randomized geo experiments is emphasized.

## Summary

The paper addresses the problem of measuring incremental Return On Ad Spend (iROAS) in the online advertising industry. The authors propose a robust nonparametric method called Trimmed Match Design (TMD) for designing reliable and cost-effective geo experiments. Geo experiments involve running randomized experiments in non-overlapping ad-targetable geographical areas. To address issues such as a small number of geos, heavy-tailed response metrics across geos due to geo heterogeneity, and varying response metrics over time, the authors extend the idea of Trimmed Match and integrate optimal subset pairing and sample splitting techniques in a novel and systematic manner. The proposed TMD method is described in detail along with its application in simulation studies and real case studies. Additionally, partial implementation code for TMD is available on GitHub [Chen et al., 2020] and previous work by Vaver & Koehler (2011), who proposed a two-stage regression model called geo-based regression (GBR) for analyzing randomized geo experiments is mentioned. The paper also highlights some open problems for future research before concluding with remarks on the effectiveness of TMD and areas for further research. Overall, this paper presents an innovative approach to address challenges associated with measuring iROAS through randomized geo experiments by offering a reliable and cost-effective solution while considering factors like small geos, heavy-tailed response metrics, and temporal variations.