Visual Semantic Image Recommendation

Guibing Guo, Yuan Meng, Yongfeng Zhang, Chunyan Han, Yanjie Li

Research output: Contribution to journalArticle


Image recommendation is an essential component of the modern online image sharing applications (e.g., Flickr), aiming to provide users with interesting images for further exploration. However, most existing approaches tend to treat the image in question as a single object, ignoring the important semantics of the sub-objects within the image. The loss of these semantic objects may lead to the misunderstanding of the user preference toward an image. In this paper, we propose a novel pairwise preference model, called Visual Semantic Model (VSM), to address this issue for a better recommendation. Specifically, we model the image representation by combining the feature embeddings of the fine-grained image objects, the weights of which may be distinct for different users. Then, we enhance the user modeling by taking into account the interacted images along with their relative importance. Two attention networks on both object and image levels are adapted to compute the weights of objects and images, respectively. The experimental results on the Flickr dataset show that our VSM model achieves significant improvements (around 9.18% on average in terms of Precision@5) over the state-of-the-art approaches in terms of the recommendation accuracy.

Original languageEnglish (US)
Article number8648433
Pages (from-to)33424-33433
Number of pages10
JournalIEEE Access
StatePublished - Jan 1 2019


All Science Journal Classification (ASJC) codes

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)


  • Image recommendation
  • attention networks
  • semantic objects

Cite this

Guo, G., Meng, Y., Zhang, Y., Han, C., & Li, Y. (2019). Visual Semantic Image Recommendation. IEEE Access, 7, 33424-33433. [8648433].