Co-clustering bipartite with pattern preservation for topic extraction

Tianming Hu, Chew Lim Tan, Yong Tang, Sam Yuan Sung, Hui Xiong, Chao Qu

Research output: Contribution to journalArticlepeer-review

5 Scopus citations

Abstract

The duality between document and word clustering naturally leads to the consideration of storing the document dataset in a bipartite. With documents and words modeled as vertices on two sides respectively, partitioning such a graph yields a co-clustering of words and documents. The topic of each cluster can then be represented by the top words and documents that have highest within-cluster degrees. However, such claims may fail if top words and documents are selected simply because they are very general and frequent. In addition, for those words and documents across several topics, it may not be proper to assign them to a single cluster. In other words, to precisely capture the cluster topic, we need to identify those micro-sets of words/documents that are similar among themselves and as a whole, representative of their respective topics. Along this line, in this paper, we use hyperclique patterns, strongly affiliated words/documents, to define such micro-sets. We introduce a new bipartite formulation that incorporates both word hypercliques and document hypercliques as super vertices. By co-preserving hyperclique patterns during the clustering process, our experiments on real-world data sets show that better clustering results can be obtained in terms of various external clustering validation measures and the cluster topic can be more precisely identified. Also, the partitioned bipartite with co-preserved patterns naturally lends itself to different clustering-related functions in search engines. To that end, we illustrate such an application, returning clustered search results for keyword queries. We show that the topic of each cluster with respect to the current query can be identified more accurately with the words and documents from the patterns than with those top ones from the standard bipartite formulation.

Original languageEnglish (US)
Pages (from-to)87-107
Number of pages21
JournalInternational Journal on Artificial Intelligence Tools
Volume17
Issue number1
DOIs
StatePublished - Feb 2008

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence

Keywords

  • Bipartite partitioning
  • Co-clustering
  • Hyperclique pattern
  • Pattern preserving
  • Topic extraction

Fingerprint

Dive into the research topics of 'Co-clustering bipartite with pattern preservation for topic extraction'. Together they form a unique fingerprint.

Cite this