# Is ChatGPT a Good Recommender? A Preliminary Study

**Authors:** Junling Liu, Chao Liu, Renjie Lv, Kang Zhou, Yan Zhang  
**arXiv:** [2304.10149](https://arxiv.org/abs/2304.10149)  
**Date:** 2023-04-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2304.10149v1/

## Key points

- ChatGPT has shown advancements in natural language processing tasks
- The paper explores using ChatGPT as a general-purpose recommendation system
- Evaluation of ChatGPT's performance as a recommendation model is done through prompts and five different recommendation scenarios
- No fine-tuning of ChatGPT is done during the evaluation process, relying solely on prompts
- Few-shot prompting is introduced to enhance ChatGPT's understanding of user needs and preferences
- Experimental results on the Amazon Beauty dataset show promising outcomes in certain tasks and baseline-level performance in others
- Human evaluations confirm that ChatGPT can comprehend provided information and generate clearer and more reasonable results
- Language models like ChatGPT have untapped potential for improving recommendation systems' performance

## Summary

The paper titled "Is ChatGPT a Good Recommender? A Preliminary Study" explores the potential of using ChatGPT, a conversational model, as a general-purpose recommendation system. The authors highlight that while traditional recommendation methods lack efficient generalization ability, ChatGPT has shown significant advancements in natural language processing tasks. To evaluate ChatGPT's performance as a recommendation model, the authors design a set of prompts and assess its capabilities in five different recommendation scenarios. Notably, they do not fine-tune ChatGPT during the evaluation process but rely solely on the prompts to convert recommendation tasks into natural language tasks. The study also introduces the concept of few-shot prompting to inject interaction information containing user potential interests. This approach aims to enhance ChatGPT's understanding of user needs and preferences. The experimental results on the Amazon Beauty dataset demonstrate promising outcomes in certain tasks and baseline-level performance in others. To further evaluate the quality of generated content by different models, human evaluations are conducted on two explainability-oriented tasks. These evaluations confirm that ChatGPT can genuinely comprehend provided information and generate clearer and more reasonable results. Overall, this preliminary study suggests that language models like ChatGPT have untapped potential for improving recommendation systems' performance and encourages researchers to delve deeper into leveraging such models to advance recommendations and contribute to this field's progress.