Illustrate your story: Enriching text with images

Sreyasi Nag Chowdhury, William Cheng, Gerard De Melo, Simon Razniewski, Gerhard Weikum

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

Human perception is known to be predominantly visual. As modern web infrastructure promoted the storage of media, the web-data paradigm shifted from text-only documents to those containing text and images. A multitude of blog posts, news articles, and social media posts exist on the Internet today as examples of multimodal stories. The manual alignment of images and text in a story is time-consuming and labor intensive. We present a web application for automatically selecting relevant images from an album and placing them in suitable contexts within a body of text. The application solves a global optimization problem that maximizes the coherence of text paragraphs and image descriptors, and allows for exploring the underlying image descriptors and similarity metrics. Experiments show that our method can align images with texts with high semantic fit, and to user satisfaction.

Original languageEnglish (US)
Title of host publicationWSDM 2020 - Proceedings of the 13th International Conference on Web Search and Data Mining
PublisherAssociation for Computing Machinery, Inc
Pages849-852
Number of pages4
ISBN (Electronic)9781450368223
DOIs
StatePublished - Jan 20 2020
Event13th ACM International Conference on Web Search and Data Mining, WSDM 2020 - Houston, United States
Duration: Feb 3 2020Feb 7 2020

Publication series

NameWSDM 2020 - Proceedings of the 13th International Conference on Web Search and Data Mining

Conference

Conference13th ACM International Conference on Web Search and Data Mining, WSDM 2020
Country/TerritoryUnited States
CityHouston
Period2/3/202/7/20

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Software
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'Illustrate your story: Enriching text with images'. Together they form a unique fingerprint.

Cite this