EP08 - Fairness and Relevance in Recommender Systems. With Dr. Theresia Veronika Rampisela
On other platforms: Web, Apple Podcast, YouTube.
Fairness (making sure every item is exposed equally) and relevance (making sure every item is relevant for the user) are 2 competing dimensions in Recommender Systems. Fairness is even difficult to define and measure, and even taking into account metrics like standard deviation or Gini index, evaluating fairness and relevance alone brings to two different "best models" which are optimized for the different dimensions.
This is what we discuss in this episode of targz, where Dr. Theresia Veronika Rampisela present a way to evaluate a model combining both fairness and relevance. Starting from the test dataset it is possible to empirically create a Pareto frontier making recommendations that progressively maximize fairness while keeping the maximum relevance. Once the frontier is created it is a matter of selecting the point that provides the target level of balance between fairness and relevance. This becomes the reference point to evaluate every model. The "best one" will be the model that is closest to the point, since it will provide the desired level of both fairness and relevance in the recommendations. The best part? The approach is model-agnostic and based only on the the test dataset and how it is selected.
That's what I learned during this interesting chat, but if you want to dig deeper in the topic checkout the paper "Joint Evaluation of Fairness and Relevance in Recommender Systems with Pareto Frontier".
That's it, see you at the next episode!