Valentin Golda, Annette Hautli-Janiszb, Katharina Holzingerc, and Mennatallah El-Assadyd
aCenter of Methods in Social Sciences, University of Göttingen, Göttingen, Germany
bDepartment of Linguistics, University of Konstanz, Konstanz, Germany
cDepartment of Politics and Public Administration, University of Konstanz, Konstanz, Germany
dDepartment of Computer and Information Science, University of Konstanz, Konstanz, Germany
The VisArgue project is an interdisciplinary collaboration of political science, linguistics and information science at the University of Konstanz (principal investigators: Miriam Butt, Katharina Holzinger, Daniel Keim), funded by the “eHumanities” research initiative of the German Federal Ministry of Education and Research (BMBF). Its main goal was the development of an automated measurement tool for the quality of deliberative communication.
The project has its theoretical foundations within the theory of deliberative democracy. Deliberative democracy promotes a form of democracy that is based on a consensus-oriented discourse instead of majority decisions and representation (Habermas, 1981; Gutmann & Thompson, 1996). At its core, the communication should be inclusive and based on extensive reasoning. Following Habermas, stakeholders participating in the discourse should be willing to adhere to “the unforced force of the better argument”.
While the empirical turn in deliberation research (Dryzek, 2000; Bächtiger, Niemeyer, Neblo, Steenbergen, & Steiner, 2010) has led to an increased understanding of deliberative decision-making, previous approaches rely on the application of manual coding schemes determining the deliberative quality within discourses (e.g. Steenbergen, Bächtiger, Spörndli, & Steiner, 2003; Wessler & Rinke, 2014). Only recently, the computational turn allows to analyze large quantities of text automatically. The VisArgue-project contributes to this turn by determining deliberative communication with automated means. We propose a novel linguistic and visual analytics toolbox that allows us to study deliberative communication in all its diverse aspects. This implies three questions:
- Which factors determine the deliberative quality of political discourse?
- How can we automatically detect and analyze these factors at the linguistic level?
- How can we use visualization in order to make patterns of deliberative communication interpretable in large amounts of text?
Based on several variants of deliberative democracy (cf. Thompson, 2008), we distinguish four dimensions that are crucial for the concept of deliberative communication (Gold et al., 2015): Participation and Inclusion, Atmosphere and Respect, Argumentation and Justification, and Accommodation and Persuasion. These four dimensions are further subdivided in different subdimensions belonging to similar theoretical concepts.
The VisArgue framework is designed to support scholars of deliberation communication in various ways. First, we propose a wide range of visual analytics tools for the interactive exploration of discourses, e.g. Lexical Episode Plots (Gold, El-Assady, & Rohrdantz, 2015) and Conversation Topic Analysis (El-Assady, Gold, Acevedo, Collins, & Keim, 2016). For a demonstration of these tools, please see http://presidential-debates.dbvis.de. Second, we propose a computational linguistic parsing system annotating various measures for the four deliberative dimensions. These measures result from the application of natural language processing tools, unsupervised content extractions, dictionary applications, and statistical analyses.
As the justification of claims is the core of the concept of deliberation, our central indicators are discourse connectors that identify causal or contrastive argumentation. We distinguish causal (weil ‘because’), conditional (wenn-dann ‘if-then’) and adversative (aber ‘but’) connectors and are able to automatically identify the type and scope of argument and even its components (such as reason or conclusion). To arrive at this annotation, several rules have to be identified that allow for the reliable disambiguation of ambiguous connectors and the argumentative phrases (Bögel, Hautli-Janisz, Sulger, & Butt, 2014).
In order to explore and interpret the various measures of deliberative communication, we propose a glyph-based visualization that is based on the annotation system but abstracts from all annotations. In general, glyphs are small iconic representations of information that are used to represent the different occurrences of the same characteristics across units in the data. The glyph shown in Figure 1 mirrors the four dimensions of deliberative communication, with each quadrant representing one dimension. Each subdimension is represented as a row and each annotation within a subdimension is represented as a small tile (the actual measure). In order to show the average length of each utterance (our unit of analysis), we include a small circular icon on the bottom left of the glyph. This is scaled to the length of the underlying utterances, indicating the relative size of its text. Moreover, to facilitate the comparison between glyphs, we account for the different types and strengths of measures by different visual mappings. For instance, the tile “average sentence complexity” in Figure 1 is marked with a plus, indicating that this turn is characterized by a higher level of sentence complexity in comparison to the other turns. Likewise, for emotions: The minus signals more negative emotions compared to the other turns.
Figure 1: Deliberation glyph with four dimensions
The glyph-based visualization supports the exploration and interpretation of deliberative communication. By providing the possibility to aggregate the glyphs with respect to different variables, e.g. the position of speakers or over topics, the turns within the discourse can be compared, resulting in conclusions with respect to the degree of deliberation. Figure 2 illustrates the mediation on “Stuttgart 21”, a conflict about re-building the rail infrastructure in Southern Germany. Four political positions are shown: pro- and contra Stuttgart 21, neutral (the mediator), and expert opinions. While the middle column illustrates the complete glyph, the right column only shows causal argumentation (green tiles) constituting of reasons and conclusions.
The overview reveals that the arbitrator provides little argumentation; the experts mostly present reasons; the pro and con parties frequently use both components. From the other measures in the left column, we can conclude that the mediator acts professionally, as he – addressing the public – uses simple and polite language. Furthermore, he interrupts other speakers, keeps them to the topics, often formulates concessions and emphasizes agreements while rarely uttering dissent. The parties to the conflict are similarly polite, but emphasize positive emotions, have a high degree of argumentative speech and add new topics to the table, thereby advancing the dialog.
Figure 2: Deliberation glyphs for the four positions in the S21 arbitration
The VisArgue framework can be applied to foster deliberation analyses. In general, the complete framework of deliberative decision-making comprises of the antecedents for deliberation (input dimension), the dynamics (throughput dimension) of deliberation, and the consequences of deliberation (output dimension). While the VisArgue project contributes to measuring variables at the throughput dimension, for deliberation to be analyzed, it is important to include both antecendents and consequences in the statistical analyses.
Here, we briefly want to describe two of our applications. First, we apply the novel measures to the verbatim minutes of simulated debates in which participants had to take unanimous decision on a contested issue (Gold & Holzinger, 2015). For each speaker, we test the effects of deliberative standards on their perception by other participants. The results show that participants are the less often perceived as constructive speakers the less someone participates. Moreover, less eloquent participants and those participants that more often express opposition are more often seen as having blocked the decision-making process.
Second, we can show how personality traits influence deliberative quality (Gold, 2016). Based on the simulated data, we see that participants scoring high on openness show less adversative argumentation. Likewise, these participants less often express conditions – contrary to participants scoring high on agreeableness. We conclude that while designing political institutions to foster deliberative communication is a good start, we need to adopt a more disaggregated focus disentangling the causal effects on the dynamics of deliberation.
Notes on contributors
Valentin Gold (PhD, Konstanz) is a postdoctoral researcher at the Center of Methods in Social Sciences at the University of Göttingen, Germany. In his research he focuses on text-mining applications and deliberative communication.
Annette Hautli-Janisz (PhD, Konstanz) is a postdoctoral researcher in the Department of Linguistics at the University of Konstanz, Germany. She has worked on various aspects of theoretical and computational linguistics, among them lexical semantics and discourse processing.
Katharina Holzinger (PhD, Augsburg) is Professor for International Politics and Conflict Management at the University of Konstanz, Germany. Her research focuses on deliberative democracy, traditional forms of governance and EU studies.
Mennatallah El-Assady is a PhD candidate in the Department of Computer and Information Science, University of Konstanz, Germany. Her research focuses on information visualization, natural language processing, and data mining.
References
El-Assady, M., Gold, V., Acevedo, C., Collins, C., & Keim, D. (2016). ConToVi: Multi-party conversation exploration using topic-space views. Computer Graphics Forum, 35(3), 431–440. https://doi.org/10.1111/cgf.12919
Bächtiger, A., Niemeyer, S., Neblo, M. A., Steenbergen, M. R., & Steiner, J. (2010). Disentangling diversity in deliberative democracy: Competing theories, their blind spots and complementarities. Journal of Political Philosophy, 18(1), 32–63. https://doi.org/10.1111/j.1467-9760.2009.00342.x
Bögel, T., Hautli-Janisz, A., Sulger, S., & Butt, M. (2014). Automatic detection of causal relations in German multilogs. In Proceedings of the EACL 2014 Workshop on Computational Approaches to Causality in Language (CAtoCL) (pp. 20–27). Gothenburg, Sweden: Association for Computational Linguistics. Retrieved from https://kops.uni-konstanz.de/handle/123456789/29255
Dryzek, J. S. (2000). Deliberative democracy and beyond: Liberals, critics, contestations. Oxford, England: Oxford University Press.
Gold, V. (2016). Personality traits and debate quality. Presented at the Symposium ”Exploring ignorance: Acquisition, selection and processing of information”, Konstanz, Germany.
Gold, V., El-Assady, M., Bögel, T., Rohrdantz, C., Butt, M., Holzinger, K., & Keim, D. (2015). Visual linguistic analysis of political discussions: Measuring deliberative quality. Digital Scholarship in the Humanities. https://doi.org/10.1093/llc/fqv033
Gold, V., & Holzinger, K. (2015). An automated text-analysis approach to measuring the quality of deliberative communication. Presented at the Annual Meeting of the American Political Science Association, San Francisco, CA.
Gold, V., Rohrdantz, C., & El-Assady, M. (2015). Exploratory text analysis using lexical episode plots. In E. Bertini, J. Kennedy, & E. Puppo (Eds.), Eurographics Conference on Visualization (EuroVis). The Eurographics Association. https://doi.org/10.2312/eurovisshort.20151130
Gutmann, A., & Thompson, D. F. (1996). Democracy and disagreement: Why moral conflict cannot be avoided in politics, and what should be done about it. Cambridge, MA: Harvard University Press.
Habermas J. (1981). Theorie des kommunikativen Handelns. Frankfurt am Main, Germany: Suhrkamp.
Steenbergen, M. R., Bächtiger, A., Spörndli, M., & Steiner, J. (2003). Measuring political deliberation: A discourse quality index. Comparative European Politics, 1(1), 21–48. https://doi.org/10.1057/palgrave.cep.6110002
Thompson, D. F. (2008). Deliberative democratic theory and empirical political science. Annual Review of Political Science, 11, 497–520. https://doi.org/10.1146/annurev.polisci.11.081306.070555
Wessler, H., & Rinke, E. M. (2014). Deliberative performance of television news in three types of democracy: Insights from the United States, Germany, and Russia. Journal of Communication, 64(5), 827–851. https://doi.org/10.1111/jcom.12115