SPEAKERS

POSTER SESSION

SCHEDULE

LOCATION

ABSTRACTS

Multimodal

Argumentation

Bridging Linguistics, Semiotics & Philosophy

Sponsored by:

Speakers

Click on photos and titles to read presentation abstracts

ASSIMAKIS TSERONIS

Which Semiotics for the Analysis and Evaluation of Multimodal Argumentation?

Hartmut STÖCKL

Moving the Multimodal into the Argumentative Limelight: Practical Suggestions for Argument Reconstruction

CHARLES FORCEVILLE

Visual and Multimodal Metaphors as Instruments to Persuade

john bateman

From Key Components to Dynamic Multimodal Discourse Semantics: Explicitating Explicitation

Gabriela Kišiček

Auditory Arguments and the Possibility of Refutation

Andrea Rocci

(Un-)demoing: Technology Demos, Hype and the Refutation of Demonstration Ad Oculos

Jean Wagemans

Identifying Multimodal Arguments

Lotte van Poppel

Jan Albert van Laar

The Argumentative Limits of Multimodal Activism

Janina Wildfeuer

Dimitris Serafis

Cohesion, Coherence, and Inference in Multimodal Argumentation: A Three-Step Bottom-Up Approach to Unveil the Argumentativity of Multimodal Artefacts

Chiara Pollaroli

Multimodality and argumentation in the design process of online multimodal rhetorical content. A case study from the Swiss Federal Office for Agriculture

Poster Session

Schedule

Speakers’ Dinner

Location

M.0074: Noordamzaal

The Noordamzaal is a lecture hall in the Munting Building (Muntinggebouw)

at Grote Kruisstraat/Nieuwe Kijk in't Jatstraat, 9712 TS Groningen.

To find the room:

From Nieuwe Kijk in't Jatstraat pass the Jantina Tammes Building (no. 68) and find the little side entrance right next to the house with the green door (no. 70), where the lady in the photo is entering.

Follow the path through the little garden to the right and follow the signs to Muntingebouw.

The first little path to the right will bring you to the Noordamzaal.

Save your spot

A unique event filled with networking, presentations, posters, and engaging conversations with the experts in the field.

Sponsored by:

Abstracts

Presentation

Which Semiotics for the Analysis and Evaluation of Multimodal Argumentation?

By Assimakis Tseronis

This presentation delves into the interdisciplinary challenge of analyzing and evaluating multimodal discourse, emphasizing the need to define arguments and understand modes. However, many scholars studying multimodal argumentation overlook the processes of multimodal meaning-making. The speaker explores various accounts of meaning-making proposed by multimodality scholars, aiming to illuminate how these perspectives can enhance the interpretation, reconstruction, and evaluation of multimodal arguments. By illustrating with examples from a corpus of images analyzed by argumentation scholars, both those referencing multimodality and those who do not, the presentation seeks to highlight the implications of different semiotic perspectives on reconstructing and evaluating multimodal arguments.

Presentation

Moving the Multimodal into the Argumentative Limelight: Practical Suggestions for Argument Reconstruction

By Hartmut Stöckl

Multimodal argument reconstruction has put the emphasis on logical form and pragmatic mechanisms. By contrast, too little attention has been devoted to the inherent properties of the modes involved. The talk proposes a three-tier heuristic for studying multimodal argumentation that involves the logic and reach of the modes, their multimodal connectedness (coherence) and the multimodal discourse structure providing the substance of the argument. The methodological suggestions are developed on plastic pollution print-ads and commercials.

Presentation

Visual and Multimodal Metaphors as Instruments to Persuade

By Charles Forceville

Aristotle's recognition of metaphors' persuasive power remains relevant today, transcending medium and genre. Metaphor interpretation involves identifying features and emotions from a source domain mapped onto a target domain. Pat Benatar's "love is a battlefield" exemplifies this, with ruthlessness and winning/losing emotions mapped onto love. Contemporary persuasive discourse, like advertising and political cartoons, often employs visual or multimodal metaphors. This paper analyzes such metaphors, focusing on visuals' role in evoking emotions (pathos) and providing proof for central claims. Examples from advertising and political cartoons will illustrate how visuals enhance metaphorical persuasion and argumentation.

Presentation

From Key Components to Dynamic Multimodal Discourse Semantics: Explicitating Explicitation

By John Bateman

Leo Groarke's framework for multimodal arguments faces criticism for its perceived bias and lack of reliability. Critics argue that identified components may reflect analysts' preferences rather than objective analysis. Proposed solutions, such as creating propositional representations of non-verbal content, are also contentious, with visual analysts rejecting direct equivalence with verbal representations. This presentation proposes leveraging multimodality theory to develop propositional models for non-verbal materials. By deriving discourse structures from material analysis, this approach aims for more reliable identification of key components, in both material evidence and explicit interpretation hypotheses. Examples illustrate this approach's effectiveness from both multimodal and argumentation perspectives.

Presentation

Auditory Arguments and the Possibility of Refutation

By Gabriela Kišiček

This paper explores the viability of auditory refutations, paralleling visual argumentation studies. It contends that sounds, like silence opposing noise, can serve as refutations, exemplified by contrasting war sounds with peaceful ones or chaos with harmony. It delves into real-life instances, such as voice identification arguments, where different aspects of the same sound are used for and against a claim. This research extends the examination of auditory arguments, probing the argumentative significance of sounds, and presents a nuanced understanding of how auditory elements contribute to the discourse.

Presentation

(Un-)demoing: Technology Demos, Hype and the Refutation of DemonstratioN Ad Oculos

By Andrea Rocci

Product and technology demonstrations, crucial for start-ups and tech investments, wield significant persuasive power. They hype investors and entice consumers, making them a vital rhetorical genre. Analyzing recent corporate examples like Nikola Motors and Google's AI, we explore demos' inferential and rhetorical impact. Demos serve as both evidence for arguments and anticipatory simulations of experience. Discussants in relevant fields counter these demos with rebuttals or undercutters, influencing ongoing arguments. We examine how demos utilize various semiotic modes and their role in shaping argumentative discourse, shedding light on their ambiguous nature as both evidence and persuasive rhetoric in the venture landscape.

Presentation

Identifying Multimodal Arguments

By Jean Wagemans

This presentation introduces the Multimodal Argument Type Identification Procedure (MATIP) to address the challenge of identifying arguments incorporating non-verbal elements like visuals. Current approaches primarily focus on verbal arguments, leaving gaps in analyzing multimodal arguments. Leveraging the Periodic Table of Arguments (PTA) framework, MATIP enables identification across verbal and non-verbal modalities through 'cross-modal mapping'. This method streamlines translating non-verbal information into verbal forms, exemplified through a sample multimodal argument analysis. MATIP offers a systematic solution for recognizing argument types in diverse modalities, bridging the gap between verbal and non-verbal elements in argumentation analysis.

Presentation

The Argumentative Limits of Multimodal Activism

By Lotte van Poppel &

Jan Albert Van Laar

Research on multimodal argumentation highlights the importance of diverse communication forms in constructing arguments. This inclusive approach acknowledges non-verbal communication's argumentative value, fostering broader understanding and representation. It encourages individuals to explore varied modes of expression, potentially bridging societal divides. Yet, it also warns against misinterpreting persuasive artifacts, which may lack substantive reasoning or exploit emotional manipulation. This presentation examines multimodal activism, such as climate and animal rights advocacy, delving into the challenges of discerning valid arguments from emotive appeals. It advocates for analytical tools to navigate this complexity, promoting balanced interpretations in the face of performative activism.

Presentation

Cohesion, Coherence, and Inference in Multimodal Argumentation: A Three-Step Bottom-Up Approach to Unveil the Argumentativity of Multimodal Artefacts

By Janina Wildfeuer &

Dimitris Serafis

This presentation introduces a three-step analytical framework to uncover argumentative inferences in multimodal artifacts. It begins by applying discourse semantics to decipher the underlying logic of these artifacts. Then, it employs social semiotic analysis to understand their meaning potential in context. Using these insights, it examines elements that strengthen the artifacts' argumentativity. This approach, based on Rigotti and Greco's Model, builds a semantic representation, interprets their meaning potential, and showcases how different modes interact within an argumentative structure. Through case analyses of multimodal news portal items, the presentation demonstrates the efficacy of this approach in revealing inferential reasoning.

Presentation

Multimodality and argumentation in the design process of online multimodal rhetorical content. A case study from the Swiss Federal Office for Agriculture

By Chiara Pollaroli

The study analyzes the design process of multimodal artefacts for social media by the Swiss Federal Office for Agriculture (FOAG), focusing on interactions, proposals, and negotiations among stakeholders with different roles and opinions. These artefacts, derived from press releases, aim to inform citizens about agricultural matters. The research investigates the semantic and pragmatic roles of multimodal elements in this design process, emphasizing their use in feedback and editing. While existing studies often scrutinize final multimodal products, little attention has been given to how multimodality influences the design process and the eventual outcome of such artefacts.

Poster

Reach Out and Touch Faith - Modes of Representation in Televangelism

By Bruna Louzada

Televangelism, a product of technological advancement, lacks linguistic analysis regarding speech-gesture ensembles employed by pastors. This study aims to fill this gap by scrutinizing Pastor Silas Malafaia's gestures during sermons, given his significant presence in Brazilian televangelism for four decades. The research focuses on how gestures create references to various entities like churchgoers, viewers, biblical characters, and the divine, illustrating narratives and parables. Through analyzing gesture units, types, and modes of representation, this research seeks to comprehend the linguistic and discursive aspects of televangelism, shedding light on the use of concrete and abstract gestures in a multimodal context.

Poster

Systematic Malicious Use of Generative Artificial Intelligence as Modus Operandi in Online Extremism

By Alexandra Kuzmina

This paper investigates the utilization of generative AI in online extremism, examining how AI-generated imagery reflects extremist tactics. It employs a theoretical framework merging Visual Criminology, Multimodal Argumentation (MA), & AI Ethics to explore how AI shapes online extremism. Visual Criminology assesses how AI development may encourage extremist behaviors by providing new propaganda tools. Through Multimodal Argumentation, it analyzes how AI-generated images convey argumentative value within extremist communities. Ethical considerations of AI use are addressed, emphasizing accountability & societal impacts.

Poster

The JOURNEY Metaphor in Huawei Mobile Phone Commercials

By Zhangjingwen Jia

This study explores Huawei mobile phone commercials' strategic use of the JOURNEY metaphor, employing Conceptual Metaphor Theory (CMT) to analyze their multimodal elements. Through visual, verbal, & auditory resources, the commercials construct the metaphorical narrative of using a Huawei phone as a journey, emphasizing progress & exploration. Visual elements play a significant role, supported by sound & music, enhancing the metaphor's communication of themes. The analysis demonstrates how advertising leverages medium-specific features to diverge from verbal metaphors, utilizing time & space compression & hyperbole. The embodied nature of the JOURNEY metaphor aligns the phone with consumer aspirations, contributing to its universal appeal in commercials.

Poster

Rethinking Visual Irony Through Echoic Attribution

By Jaime Salvador-Grande

Visual & multimodal irony, despite receiving less scholarly focus than verbal irony, are significant areas of study. Current analyses draw from echoic & pretense theories of verbal irony. Scott suggests visual irony arises from incongruities within or between an image & reality, while Currie suggests caution, arguing many instances are depictions of ironic situations rather than communicative irony. This poster proposes integrating Currie's distinction into Wilson & Sperber's echoic theory. It suggests visuals convey communicative irony when the creator's intended meaning is discernible, enhancing interpretative scope by incorporating the concept of attribution. This approach extends understanding of visual & multimodal utterances beyond pretense theory.

Poster

How to Analyze Multimodal Argumentation in Short Videos? Merging Multimodality and Argumentation Theory

By Maciej Grzenkowicz

TikTok & other video applications posed a particular challenge to the argumentation theorists: with unprecedented semiotic richness of posts published on these platforms, a particularly fine-grained tool for analysis was needed. I propose merging two, until now quite distinct, disciplines - argumentation theory & linguistics-based multimodality - to address this issue. I start by presenting a multi-level, multimodal annotation scheme for short videos. Afterward, I propose a semiotic approach to argumentation based on the notion of Searle’s speech act adjusted for multimodal purposes. Finally, I show how these two approaches can be successfully combined by analyzing an inherently argumentative TikTok video genre: fact-checking videos.

Poster

The Speech Act Theoretic Approach to Recognizing Visual Arguments: Proportionality and Intention

By Bita Heshmati

Recognizing a visual argument in a particular context involves determining whether the image(s) used in that context serve an argumentative function, for example, whether the relevant image is used for establishing the acceptability of a proposition expressed by the arguer. A crucial question is, how can we determine whether a particular image is used argumentatively or not? In my paper, I attempt to answer this question by employing Speech Act Theory (Searle 1980, van Eemeren and Grootendorst 1994). Following this model, I conceptualize images as (indirect) speech acts that can perform acts of asserting, claiming, etc., non-verbally. Accordingly, I formulate the recognizability conditions as follows: 1. The image(s) should consist of assertives in which propositions are expressed (Propositionality Condition) 2. Advancing image(s) counts as an attempt by the arguer to justify a proposition to the viewer, i.e., to convince the viewer of the acceptability of that proposition (Intentionality Condition). These conditions jointly require that the arguer uses images to convey propositions which can be identified by the viewer.