# Real-World Recommender Systems for Academia: The Pain and Gain in Building, Operating, and Researching them [Long Version]

**Authors:** Joeran Beel, Siddharth Dinesh  
**arXiv:** [1704.00156](https://arxiv.org/abs/1704.00156)  
**Date:** 2017-04-01  
**Summary page:** https://www.summarizepaper.com/en/paper/1704.00156v1/

## Key points

- Non-reproducible research results
- Dealing with noisy data
- Determining the optimal number and frequency of recommendations
- Three research-article recommender systems built by the authors over six years
- Lack of guidance from existing literature in identifying effective recommendation approaches
- Difficulties encountered in creating a randomization engine for A/B tests
- Low data quality affecting bibliometrics calculations and evaluation process
- Experiments yielding disappointing results and reasons behind them
- Statistics on researcher interest in recommendation dataset
- Insights into skill requirements, limitations of existing literature, experimental design issues, data quality concerns, and researcher engagement with recommendation datasets.

## Summary

The paper titled "Real-World Recommender Systems for Academia: The Pain and Gain in Building, Operating, and Researching them [Long Version]" by Joeran Beel and Siddharth Dinesh discusses the challenges faced in research on recommender systems. The authors highlight major obstacles such as non-reproducible research results, dealing with noisy data, and determining the optimal number and frequency of recommendations. Over a period of six years, the authors built three research-article recommender systems for digital libraries and reference managers. They conducted extensive research on these systems and share their experiences in this paper. One of the key points discussed is the lack of guidance from existing literature in identifying effective recommendation approaches. The authors also explain the difficulties encountered in creating a randomization engine to run A/B tests for evaluating recommender system performance. Additionally, they address how low data quality affects bibliometrics calculations which further complicates the evaluation process. Furthermore, the paper explores why some of their experiments yielded disappointing results. The authors provide statistics on the level of interest shown by researchers in their recommendation dataset. Overall, this paper provides valuable insights into the challenges faced when building and operating recommender systems for academia. It sheds light on various aspects such as skill requirements, limitations of existing literature, experimental design issues, data quality concerns, and researcher engagement with recommendation datasets.