ICML 2011 workshop on unsupervised and transfer learning

Self-reflective Multi-task Gaussian Process

Kohei Hayashi, Takashi Takenouchi
Nara Institute of Science and Technology, Japan

Ryota Tomioka, Hisashi Kashima
University of Tokyo, Japan

Multi-task learning aims at transferring knowledge between similar tasks. The multi-task Gaussian process framework of Bonilla et al. (2008) models (incomplete) responses of C data points for R tasks by a Gaussian process; the covariance function is defined as the product of a covariance function on input-dependent features and the inter-task covariance matrix (empirically estimated as a model parameter). We propose a new model in which we incorporate similarities based on the observed responses, which allows for the representation of much more complex data structures. The proposed approach enables us to construct covariance matrices via kernel functions even when additional information (e.g., the inputdependent features) are not given. We also propose an efficient conjugate-gradient-based
algorithm for prediction, which solves an R ~ 900 and C ~ 1600 problem in roughly 10 minutes. Finally, we apply our model to the Movielens 100k dataset and show that the proposed method achieves the best prediction accuracy on the dataset.