EAGER: SaTC: Early-Stage Interdisciplinary Collaboration: Fair and Accurate Information Quality Assessment Algorithm

Prevalence of poor-quality information in cyberspaces poses threats to civic society. To increase information quality, multiple automated algorithms for undertaking quality assessment of online information have been proposed. However, the fairness and performance of these algorithms across political and policy opinions has been challenged, undermining trust in such systems. Through a unique early-stage interdisciplinary collaboration that brings together experts from the fields of information science, computer science, communication, political science, and journalism, this project will develop accurate and fair information quality assessment algorithms, while also gleaning deeper insight into the nature of information being utilized across the ideological spectrum. The proposed research advances the science of information and will offer insights to organizations that aim to undertake automated information quality assessment, ultimately allowing for the creation of safer and trustworthy cyberspaces. The proposed project will include: (1) the creation of a large article dataset that has been robustly labeled for both quality and political ideological alignment, (2) an audit of multiple existing information quality assessment algorithms to assess their accuracy and fairness, (3) a systematic post-hoc inductive analysis of the content mislabeled by these algorithms, and (4) modification of existing algorithms to support fairer and more accurate information quality assessment. These four phases will build upon each other, leveraging the contributions of each discipline and will provide a new interdisciplinary model for SaTC-related research. This project will provide interdisciplinary training to graduate students, mentoring them in diverse methods and laying the groundwork for long-term interdisciplinary research. This project also will help broaden participation in data science professions. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.