The structure of broad topics on the web

Soumen Chakrabarti, Mukul M. Joshi, Kunal Punera, David M. Pennock

Research output: Chapter in Book/Report/Conference proceedingConference contribution

68 Scopus citations

Abstract

The Web graph is a giant social network whose properties have been measured and modeled extensively in recent years. Most such studies concentrate on the graph structure alone, and do not consider textual properties of the nodes. Consequently, Web communities have been characterized purely in terms of graph structure and not on page content. We propose that a topic taxonomy such as Yahoo! or the Open Directory provides a useful framework for understanding the structure of content-based clusters and communities. In particular, using a topic taxonomy and an automatic classifier, we can measure the background distribution of broad topics on the Web, and analyze the capability of recent random walk algorithms to draw samples which follow such distributions. In addition, we can measure the probability that a page about one broad topic will link to another broad topic. Extending this experiment, we can measure how quickly topic context is lost while walking randomly on the Web graph. Estimates of this topic mixing distance may explain why a global PageRank is still meaningful in the context of broad queries. In general, our measurements may prove valuable in the design of community-specific crawlers and link-based ranking systems.

Original languageEnglish (US)
Title of host publicationProceedings of the 11th International Conference on World Wide Web, WWW '02
Pages251-262
Number of pages12
DOIs
StatePublished - 2002
Externally publishedYes
Event11th International Conference on World Wide Web, WWW '02 - Honolulu, HI, United States
Duration: May 7 2002May 11 2002

Publication series

NameProceedings of the 11th International Conference on World Wide Web, WWW '02

Other

Other11th International Conference on World Wide Web, WWW '02
CountryUnited States
CityHonolulu, HI
Period5/7/025/11/02

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Computer Science Applications

Keywords

  • Social network analysis
  • Web bibliometry

Fingerprint Dive into the research topics of 'The structure of broad topics on the web'. Together they form a unique fingerprint.

Cite this