Visual Semantic Image Recommendation

Guibing Guo, Yuan Meng, Yongfeng Zhang, Chunyan Han, Yanjie Li

Research output: Contribution to journalArticle

Abstract

Image recommendation is an essential component of the modern online image sharing applications (e.g., Flickr), aiming to provide users with interesting images for further exploration. However, most existing approaches tend to treat the image in question as a single object, ignoring the important semantics of the sub-objects within the image. The loss of these semantic objects may lead to the misunderstanding of the user preference toward an image. In this paper, we propose a novel pairwise preference model, called Visual Semantic Model (VSM), to address this issue for a better recommendation. Specifically, we model the image representation by combining the feature embeddings of the fine-grained image objects, the weights of which may be distinct for different users. Then, we enhance the user modeling by taking into account the interacted images along with their relative importance. Two attention networks on both object and image levels are adapted to compute the weights of objects and images, respectively. The experimental results on the Flickr dataset show that our VSM model achieves significant improvements (around 9.18% on average in terms of Precision@5) over the state-of-the-art approaches in terms of the recommendation accuracy.

Original languageEnglish (US)
Article number8648433
Pages (from-to)33424-33433
Number of pages10
JournalIEEE Access
Volume7
DOIs
StatePublished - Jan 1 2019

Fingerprint

Semantics

All Science Journal Classification (ASJC) codes

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)

Keywords

  • Image recommendation
  • attention networks
  • semantic objects

Cite this

Guo, G., Meng, Y., Zhang, Y., Han, C., & Li, Y. (2019). Visual Semantic Image Recommendation. IEEE Access, 7, 33424-33433. [8648433]. https://doi.org/10.1109/ACCESS.2019.2900396
Guo, Guibing ; Meng, Yuan ; Zhang, Yongfeng ; Han, Chunyan ; Li, Yanjie. / Visual Semantic Image Recommendation. In: IEEE Access. 2019 ; Vol. 7. pp. 33424-33433.
@article{8767ba1fcdca47779b75b671f1811339,
title = "Visual Semantic Image Recommendation",
abstract = "Image recommendation is an essential component of the modern online image sharing applications (e.g., Flickr), aiming to provide users with interesting images for further exploration. However, most existing approaches tend to treat the image in question as a single object, ignoring the important semantics of the sub-objects within the image. The loss of these semantic objects may lead to the misunderstanding of the user preference toward an image. In this paper, we propose a novel pairwise preference model, called Visual Semantic Model (VSM), to address this issue for a better recommendation. Specifically, we model the image representation by combining the feature embeddings of the fine-grained image objects, the weights of which may be distinct for different users. Then, we enhance the user modeling by taking into account the interacted images along with their relative importance. Two attention networks on both object and image levels are adapted to compute the weights of objects and images, respectively. The experimental results on the Flickr dataset show that our VSM model achieves significant improvements (around 9.18{\%} on average in terms of Precision@5) over the state-of-the-art approaches in terms of the recommendation accuracy.",
keywords = "Image recommendation, attention networks, semantic objects",
author = "Guibing Guo and Yuan Meng and Yongfeng Zhang and Chunyan Han and Yanjie Li",
year = "2019",
month = "1",
day = "1",
doi = "10.1109/ACCESS.2019.2900396",
language = "English (US)",
volume = "7",
pages = "33424--33433",
journal = "IEEE Access",
issn = "2169-3536",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

Guo, G, Meng, Y, Zhang, Y, Han, C & Li, Y 2019, 'Visual Semantic Image Recommendation', IEEE Access, vol. 7, 8648433, pp. 33424-33433. https://doi.org/10.1109/ACCESS.2019.2900396

Visual Semantic Image Recommendation. / Guo, Guibing; Meng, Yuan; Zhang, Yongfeng; Han, Chunyan; Li, Yanjie.

In: IEEE Access, Vol. 7, 8648433, 01.01.2019, p. 33424-33433.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Visual Semantic Image Recommendation

AU - Guo, Guibing

AU - Meng, Yuan

AU - Zhang, Yongfeng

AU - Han, Chunyan

AU - Li, Yanjie

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Image recommendation is an essential component of the modern online image sharing applications (e.g., Flickr), aiming to provide users with interesting images for further exploration. However, most existing approaches tend to treat the image in question as a single object, ignoring the important semantics of the sub-objects within the image. The loss of these semantic objects may lead to the misunderstanding of the user preference toward an image. In this paper, we propose a novel pairwise preference model, called Visual Semantic Model (VSM), to address this issue for a better recommendation. Specifically, we model the image representation by combining the feature embeddings of the fine-grained image objects, the weights of which may be distinct for different users. Then, we enhance the user modeling by taking into account the interacted images along with their relative importance. Two attention networks on both object and image levels are adapted to compute the weights of objects and images, respectively. The experimental results on the Flickr dataset show that our VSM model achieves significant improvements (around 9.18% on average in terms of Precision@5) over the state-of-the-art approaches in terms of the recommendation accuracy.

AB - Image recommendation is an essential component of the modern online image sharing applications (e.g., Flickr), aiming to provide users with interesting images for further exploration. However, most existing approaches tend to treat the image in question as a single object, ignoring the important semantics of the sub-objects within the image. The loss of these semantic objects may lead to the misunderstanding of the user preference toward an image. In this paper, we propose a novel pairwise preference model, called Visual Semantic Model (VSM), to address this issue for a better recommendation. Specifically, we model the image representation by combining the feature embeddings of the fine-grained image objects, the weights of which may be distinct for different users. Then, we enhance the user modeling by taking into account the interacted images along with their relative importance. Two attention networks on both object and image levels are adapted to compute the weights of objects and images, respectively. The experimental results on the Flickr dataset show that our VSM model achieves significant improvements (around 9.18% on average in terms of Precision@5) over the state-of-the-art approaches in terms of the recommendation accuracy.

KW - Image recommendation

KW - attention networks

KW - semantic objects

UR - http://www.scopus.com/inward/record.url?scp=85063912035&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063912035&partnerID=8YFLogxK

U2 - 10.1109/ACCESS.2019.2900396

DO - 10.1109/ACCESS.2019.2900396

M3 - Article

AN - SCOPUS:85063912035

VL - 7

SP - 33424

EP - 33433

JO - IEEE Access

JF - IEEE Access

SN - 2169-3536

M1 - 8648433

ER -

Guo G, Meng Y, Zhang Y, Han C, Li Y. Visual Semantic Image Recommendation. IEEE Access. 2019 Jan 1;7:33424-33433. 8648433. https://doi.org/10.1109/ACCESS.2019.2900396