Commenting Systems for News – CommentIQ

Online news outlets are concerned with the civility and quality of the dialogue that emerges in news comments. Incivility can lead to backfire effects like further polarization on issues. But there’s two sides to the comment quality issue. What about the really high quality comments that get made? (Yes, there are many, but they can get drowned out). Research suggests that by curating and highlighting such high quality comments this can signal norms and expectations for comment discourse, thus improving the long term quality and viability of online discussion of important civic issues. The New York Times commenting system currently supports this idea of “NYT Picks”, editor’s selections of the “most interesting and thoughtful” comments which are available via a filtered tab within their commenting interface.

The Knight Foundation Funded CommentIQ project is about expanding the idea of “editor’s selections” to make it easier for community managers and moderators in a range of news organizations to quickly identify high quality comments that they can then highlight on their sites. Initial research, published and cited below, has looked at over 330,000 comments and 12,500 editor’s selections to identify textual features of comments that correlate to a comment being chosen by an editor as a “NYT Pick”. Ongoing research is evaluating 8 other criteria that help describe higher quality comments, and is looking at ways to operationalize these criteria algorithmically. Ultimately all of this data science will be made available as an API service, and will be highlighted in a visual analytic prototype interface that moderators can evaluate for the task of high quality comment selection.

Papers

  • D. Park, S. Sachar, N. Diakopoulos, and N. Elmqvist. Supporting Comment Moderators in Identifying High Quality Online News Comments. Proc. Conference on Human Factors in Computing Systems (CHI). 2016. [PDF]
  • N. Diakopoulos. Picking the NYT Picks: Editorial Criteria and Automation in the Curation of Online News Comments. #ISOJ Journal. April, 2015. [PDF]
  • N. Diakopoulos. The Editor’s Eye: Curation and Comment Relevance on the New York Times. Proc. Conference on Computer Supported Cooperative Work (CSCW). March, 2015. [PDF]
  • J. Hullman, N. Diakopoulos, E. Momeni, E. Adar. Content, Context, and Critique: Commenting on a Data Visualization Blog. Proc. Conference on Computer Supported Cooperative Work (CSCW). March, 2015. [PDF]