Wide & Deep Learning for Recommender Systems

The paper discusses the use of generalized linear models with nonlinear feature transformations for large-scale regression and classification problems with sparse inputs. These models are effective in memorizing feature interactions through cross-product feature transformations, but generalization requires more feature engineering effort. On the other hand, deep neural networks can generalize better to unseen feature combinations by learning low-dimensional dense embeddings for sparse features. However, when user-item interactions are sparse and high-rank, deep neural networks with embeddings can over-generalize and recommend less relevant items. To address these challenges, the authors propose a method called Wide & Deep learning which combines wide linear models and deep neural networks. The wide linear models focus on memorization of feature interactions while the deep neural networks capture generalization through learned embeddings. The system was evaluated on Google Play, a commercial mobile app store with over one billion active users and one million apps. Online experiment results showed that Wide & Deep significantly increased app acquisitions compared to models using only wide or deep components. The authors have also open-sourced their implementation in TensorFlow. Overall, this paper presents an approach that combines the benefits of memorization and generalization for recommender systems using both wide linear models and deep neural networks. The experimental results demonstrate its effectiveness in improving app acquisitions in a real-world setting.

Wide & Deep Learning: Combining Memorization and Generalization for Recommender Systems

Recommender systems are used to personalize user experiences by recommending content such as products, services, or information that is tailored to the individual’s interests. In order to make accurate recommendations, these systems need to be able to capture both memorization of feature interactions and generalization of unseen feature combinations. Traditional methods such as generalized linear models (GLMs) can effectively memorize feature interactions through cross-product feature transformations but require more effort in terms of feature engineering for generalization. On the other hand, deep neural networks (DNNs) can learn low-dimensional dense embeddings for sparse features which allow them to better generalize but they tend to overgeneralize when user-item interactions are sparse and high-rank.

In this paper, the authors propose a method called Wide & Deep learning which combines wide linear models with deep neural networks in order to address these challenges. The wide linear models focus on memorizing feature interactions while the deep neural networks capture generalization through learned embeddings. The system was evaluated on Google Play, a commercial mobile app store with over one billion active users and one million apps. Online experiment results showed that Wide & Deep significantly increased app acquisitions compared to models using only wide or deep components. The authors have also open sourced their implementation in TensorFlow so it can be easily adapted by other developers looking for an effective recommender system solution.

Background

Generalized Linear Models (GLMs) are widely used in large scale regression and classification problems due to their scalability and effectiveness at capturing nonlinear relationships between variables through nonlinear transformations such as polynomial expansions or cross product features like pairwise interaction terms between categorical variables. However, GLMs require more effort when it comes to generalizing beyond seen data points since they rely heavily on manually engineered features rather than learning from data itself like DNNs do with their low dimensional dense embeddings for sparse inputs which allows them better generalize unseen data points without manual intervention required by GLMs.

On the other hand, DNNs tend not perform well when user item interactions are sparse and high rank due its tendency towards overgeneralizing which leads less relevant items being recommended even though they may have been seen before during training process . To address this issue ,the authors proposed a hybrid approach combining both GLM's ability at memorizing complex relationships between variables along with DNN's capability at capturing low dimensional dense embedding representations of sparse input features .

Methodology

The proposed method combines two components : wide linear model component focusing on memorizing complex relationships between variables along with a deep neural network component responsible for capturing low dimensional dense embedding representations of sparse input features . The idea behind this approach is that while each component has its own strengths , together they should be able provide more accurate predictions than either could alone . This combination is referred as “wide & deep learning” because it leverages both aspects of machine learning -memorization(wide)and generaliztion(deep)-to create an effective recommendation system .

The authors implemented their proposed method using Google Play dataset consisting of 1 billion active users ,1 million apps ,and millions of ratings given by users about different apps . They then tested performance against baseline methods including logistic regression model only using wide components ,a feedforward network only using deep components ,and finally a combined model using both wide&deep components . Results showed that combined model outperformed all three baselines significantly increasing app acquisition rate compared against all three baselines proving effectiveness of proposed approach in real world setting .

Conclusion

Overall ,this paper presents an approach combining benefits from both traditional GLM based approaches focusing on memorizing complex relationships between variables along with modern DNN based approaches capable at capturing low dimensional dense embedding representations allowing them better generalize unseen data points without manual intervention required by GLMs . Experimental results demonstrate its effectiveness improving app acquisitions in real world setting making it attractive option for developers looking build efficient recommendation systems leveraging power from both worlds -traditional ML algorithms along side modern DL techniques-.