Résumé : Previous highly scalable one-class collaborative filtering methods such as Projected Linear Recommendation (PLRec) have advocated using fast randomized SVD to embed items into a latent space, followed by linear regression methods to learn personalized recommendation models per user. Unfortunately, naive SVD embedding methods often exhibit a popularity bias that skews the ability to accurately embed niche items. To address this, we leverage insights from Noise Contrastive Estimation (NCE) to derive a closed-form, efficiently computable "depopularized" embedding. While this method is not ideal for direct recommendation using methods like PureSVD since popularity still plays an important role in recommendation, we find that embedding followed by linear regression to learn personalized user models in a novel method we call NCE-PLRec leverages the improved item embedding of NCE while correcting for its popularity unbiasing in final recommendations. An analysis of the recommendation popularity distribution demonstrates that NCE-PLRec uniformly distributes its recommendations over the popularity spectrum while other methods exhibit distinct biases towards specific popularity subranges, thus artificially restricting their recommendations. Empirically, NCE-PLRec outperforms state-of-the-art methods as well as various ablations of itself on a variety of large-scale recommendation datasets.