Modern session-based deep recommender systems can be somehow limited by the softmax bottleneck like their language model cousins — TL;DR: The traditional softmax is limited in its capacity to fully model tasks like natural language that are highly context-dependent. This limit in expressiveness called the Softmax bottleneck can be described through the lenses of matrix factorization and the study of the resulting matrix ranks.

How to reduce the offline-online evaluation gap — Introduction In this tutorial, I will give some rationales about why one should care about off-policy evaluation. Then I’ll introduce the right amount of theory on the matter. I will not dive too deep into the latest methods and stop on what works well empirically. …

Heavy ranking model strategy and design — Multi-gate Mixture-of-Experts In the first post of this series, we learned about the importance of adopting a multi-stage recommendation strategy, especially for companies with large item catalogs. We learned what candidate generation was and implemented one of its famous representatives: the two-tower model…

Implementation of the two-tower model and its application to H&M data — This blog post is the follow-up on part 1.1 where we explained the two-stage recommendation process with a special emphasis on the candidate generation step. I encourage you to read this article first if you didn’t already. …

One of the key ingredients of CatBoost explained from the ground up — CatBoost is a “relatively” new package developed by Yandex researchers. It is pretty popular right now, especially in Kaggle competitions where it generally outperforms other gradient tree boosting libraries. Among other ingredients, one of the very cool feature of CatBoost is that it handles categorical variables out of the box…

A friendly but rigorous explanation — This article is intended for either students trying to break into data science or professionals in need of a refresher on boosting and gradient boosting. There are already quite a lot of materials regarding this topic on the web but not many include graphical visualization of the learning process. So…

A complete tutorial — In my last article, I wrote a detailed explanation of the Gaussian Mixture Model (GMM) and the way it is trained using the Expectation-Maximization (EM) algorithm. This time, I wanted to show that a mixture model is not necessarily a mixture of Gaussian densities. It can be a mixture of…

This article is an extension of “Gaussian Mixture Models and Expectation-Maximization (A full explanation)”. If you didn’t read it, this article might not be very useful. The goal here is to derive the closed-form expressions necessary for the update of the parameters during the Maximization step of the EM algorithm…

In the previous article, we described the Bayesian framework for linear regression and how we can use latent variables to reduce model complexity. In this post, we will explain how latent variables can also be used to frame a classification problem, namely the Gaussian Mixture model (or GMM in short)…