# LLM-powered Data Augmentation for Enhanced Crosslingual Performance

**Authors:** Chenxi Whitehouse, Monojit Choudhury, Alham Fikri Aji  
**arXiv:** [2305.14288](https://arxiv.org/abs/2305.14288)  
**Date:** 2023-05-23  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2305.14288v1/

## Key points

- The paper addresses the challenge of data scarcity in crosslingual commonsense reasoning datasets.
- Large Language Models (LLMs) are explored for data augmentation, including Dolly-v2, StableVicuna, ChatGPT, and GPT-4.
- Three low-resource datasets are augmented: XCOPA, XWinograd, and XStoryCloze.
- Fine-tuning smaller crosslingual models mBERT and XLMR using synthesised data is assessed for effectiveness.
- Incorporating data generated by LLMs consistently improves performance compared to baseline models.
- GPT-4 generates the best synthetic data compared to other models such as ChatGPT and Dolly-v2.
- LLMs like ChatGPT and GPT-4 excel at generating natural text in most languages except Tamil.
- Multilingual scaling through fine-tuning smaller models using synthetic data is showcased practically.
- The study provides an extensive analysis of various target languages' effects on model performance.
- Synthesised datasets are released for public use and reproducibility.

## Summary

This paper addresses the challenge of data scarcity in crosslingual commonsense reasoning datasets by exploring the potential of leveraging Large Language Models (LLMs) for data augmentation. The authors employ several LLMs, including Dolly-v2, StableVicuna, ChatGPT, and GPT-4 to augment three low-resource datasets: XCOPA, XWinograd, and XStoryCloze. They then assess the effectiveness of fine-tuning smaller crosslingual models, mBERT and XLMR, using the synthesised data. The study compares the performance of training with data generated in English and target languages as well as translating English-generated data into target languages. The experiments reveal that incorporating data generated by LLMs consistently improves performance compared to baseline models. Training on synthetic data generated by GPT-4 improves performance consistently compared to other models such as ChatGPT and Dolly-v2. The authors also conduct a human evaluation of the naturalness and logical coherence of the generated examples for different languages. The results show that LLMs like ChatGPT and GPT-4 excel at generating natural text in most languages except Tamil. Moreover, ChatGPT trails behind in generating plausible alternatives compared to the original dataset while GPT-4 demonstrates competitive logic consistency in synthesised data. In addition to augmenting low-resource datasets using LLMs, this work contributes to expanding NLP beyond English by showcasing practical value in multilingual scaling through fine-tuning smaller models using synthetic data. The study also provides an extensive analysis of various target languages' effects on model performance and releases synthesised datasets for public use and reproducibility. Previous studies have shown that zero-shot cross-lingual transfer can be effective when dealing with limited resources; however, this study shows that employing LLMs for data augmentation can improve model performance significantly. Multilingual Data Augmentation has been explored previously; however, this study provides a comprehensive analysis of the effects of various target languages on model performance. Overall, this paper's contributions include augmenting low-resource datasets using LLMs, showcasing practical value in multilingual scaling through fine-tuning smaller models using synthetic data, and providing an extensive analysis of the effects of various target languages on model performance. The authors release synthesised datasets for public use and reproducibility.