As part of the larger transformation of public communication in the digital age, professional journalists are facing an increasing amount of audience feedback, e.g. in forums, comments sections, and social media. In pre-digital times, conversations among audience members about mass media content remained largely invisible to journalists, with the exception of letters or calls to the editor. Today, the conversations of “the people formerly known as the audience” (Jay Rosen) are becoming visible to journalists, but also to other users, fundamentally changing how today’s journalists and their audiences perceive, use, and manage this kind of feedback.
Most (online) newsrooms will consider comment sections and other features for audience feedback mandatory. However, newsrooms differ regarding how they manage these spaces, how they engage their users, and how they make use of the feedback for their own journalistic reporting – not the least because the manual handling and summarising of comments by journalists or dedicated social media editors is time consuming, while a fully automated analysis is expensive and error-prone. Accordingly, the development of tools to assist journalists in analysing, filtering, and summarising user-generated content has been identified as a main challenge for news organisations.
The Hans-Bredow-Institut works together with the Department of Informatics of Universität Hamburg in order to develop a framework that supports journalist to analyse, filter, and summarise user-generated content. This framework enables them to carry out a systematic, semi-automated analysis of audience feedback to better reflect the voice of users, mitigate the analysis efforts, and help journalist in generating new content from the user comments. With the framework journalists can create different samples of user comments, configure the questions they want to answer from the comments, and assign the question-answering task to “human coders” from the crowd.
The framework uses machine learning and natural language processing techniques in combination with manual content analysis (peer coding) and crowdsourcing to automatically filter spams, distinguish between praise and criticism, and cluster the comments into customisable categories. Moreover, journalists can create basic summaries about the comments such as how many users were for or against a particular position. As part of the project, we will (a) discuss and develop the framework requirements with journalists and (b) evaluate the framework in a concrete use case with a large German online news site.
The requirements for such a system will be specified together with journalists in the course of the project. Furthermore, it will be tested within the scope of a certain case on a big German news website.
