Multitask-learning in recommendation system
In the Recommendation system domain, especially for ranking, it is very common to train a multi-task learning model (MTL) to learn different objectives at once so that the model could learn:
- Balance different business need and objective: E.g. balancing the CTR target vs the watching/reading time target. (Click-bait items vs meaningful content)
- Extract useful features from different label perspectives: E.g. insights for the high CTR but low watching/reading time vs high CTR and high watching/reading time
Therefore, we need a dataset that contains multiple labels to enable us evaluating different MTL algorithms. Further, an ideal dataset should be constructed from the real world user logs so that the evaluation is based on the real world distribution instead of a synthesized/hypothesized distribution.
Ali-CCP dataset
Fortunately, Alibaba has published such dataset: Alibaba Click and Conversion Prediction (Ali-CCP, link ) It was introduced in the well-known ESMM paper. This dataset is built from the real world traffic logs in Taobao and it is a loarge scale dataset. (~42M for train and eval respectively)
The data source illustration is as the below screenshot (shared from the Ali-CCP website link) 