Toward a Framework for Immersive Counter-Storytelling in Contested Heritage

Sarah Brown, Department of Computer and Information Science and Engineering, University of Florida, Gainesville, USA ORCID iD

October 31, 2025

DOI: https://doi.org/10.62937/JIN.2025.GRAD.4910

Abstract

Applied IDN research in health and education lacks consistent, rigorous evaluation, making it hard to link specific design features to outcomes like immersion, engagement, or learning. A survey of 66 studies found only 17 with evaluations, which were often heterogeneous in design and rarely isolated the characteristics of IDN, highlighting the evidence gap. The paper advocates richer, in-experience methods (e.g., screen recording, action logging, retrospective video interviews) and qualitative approaches to surface players’ internal processes, complemented by quantitative measures. It also acknowledges the trade-offs between time and generalizability, and highlights community efforts.

Keywords

Evaluation Methods, Health, Education, Immersion, Engagement, Qualitative Research, Retrospective Interviews

Arguing New Methodologies for the Study of Applied IDNs

1. Introduction

Throughout the scholarship on interactive digital narratives (IDNs) for health and education, there is an issue of a lack of consistent, if any, evaluation. As a result, there is no consensus as to what aspects of interactive stories incite immersion, engagement, or foster the goals of behavior change or knowledge gain. Further complicating things, some aspects exist intangibly within the design of the narrative itself and can be difficult to isolate through experimental design. If researchers and practitioners in the field cannot consistently articulate which aspects of IDNs lead to specific outcomes, it will be difficult for the area to secure a solid foundation upon which to grow and make strong arguments for IDN interventions to stakeholders in applied domains. After surveying the literature on IDNs for health and education, it became clear to me that as a field we need improved and newly developed evaluation methods, particularly ones that better assess the experience of the participant during their time spent with an IDN, as opposed to strictly after.

2. Survey of Applied IDNs

To better understand the state of the literature on IDNs for health and education, I conducted a systematic survey of 66 publications which investigated the use of IDNs in the domains of health and education, as part of my PhD qualifying exam. The goals of this survey were twofold: (i) to uncover the reasons WHY we design IDNs for health and education – reviewing the different theoretical foundations and justifications for these interventions, and (ii) understanding HOW we design IDNs for health and education. Of the 66 papers surveyed, only 17, or ~25.8% conducted an evaluation of the proposed IDN intervention. The rigor of these 17 evaluations varied greatly, with sample sizes ranging from n=2 to n=243, for an average of ~54 (excluding an outlier study which had an n of 1718). Of these evaluations, 6 held no comparison to another intervention, and among those that did, many only compared their intervention to a non-IDN equivalent. Very few of these experiments were designed to isolate characteristics of the given IDN artifact as independent variables.

3. Discussion

Due to lack of concrete findings within the literature, we cannot currently make strong claims on what aspects of IDNs result in which benefits or impacts. Furthermore, much of the effects of applied IDNs are lost within the player’s subjective experience during their playthrough, which is often ignored or glanced over in existing evaluations. I propose the development and use of methodologies which uncover richer insight into the internal processes the player undergoes as they experience an applied IDN. Comprehensive qualitative techniques which aim to piece apart the internal experience of the player may be the key to achieving greater understanding of the specific influences and impacts of IDN interventions. An example of a paper which achieves this is that of Echiverri and Wei, who used Interpretive Phenomenological Analysis (IPA) to construct a detailed model of the relationships between different aspects of the IDN and the players’ experience (2023).

More nuanced insights can be uncovered by capturing as much of the experience of playing through an IDN as possible, which can still be later supported by quantitative results. Greater use of screen-recording, action logging, or other experience capture techniques as afforded by a given IDN’s platform can provide us with richer data with which to evaluate participant experiences. While techniques such as a traditional think-aloud procedure can prove too distracting for immersive interactive experiences, methods such as retrospective video interviews can still provide greater detail than other post-hoc techniques, as they allow the researcher to retroactively ‘step through’ the participant’s experience with them.

The above arguments are not without limitations – the methodologies proposed above are all time-consuming and require intensive analysis. Effectively extracting internal insights from participants is a non-trivial task and can place too high of a metacognitive demand. Lastly, the generalizability of qualitative results will always be limited, but so is the capability of quantitative methods in capturing subjective, internal experiences and outcomes. We need to continue to have serious discussions as a research community on the appropriate methodologies with which to assess IDNs for health and education. Piecing apart such a complex web of factors is no small task but is essential for the field to move on from high-level insights across disparate, one-shot interventions. This work has already been started by INDCOR, an EU cost-action on interactive narrative design for complexity representations, in their white paper on IDN evaluation methodologies (Roth et al., 2023). Additionally, as a result of this short presentation, a workshop is now being planned for ICIDS 2025 to continue this important discussion among researchers in the field.

References

  1. Echeverri, D., & Wei, H. (2023). Exploring the experience with tangible interactive narrative: Authoring and evaluation of Letters to José. Entertainment Computing, 44, 100535.
  2. Roth, C., Pitt, B., Šķestere, L., Barbara, J., Bakk, A. K., Dunlop, K., ... & Schlauch, M. (2023). INDCOR white paper 4: Evaluation of Interactive Narrative Design For Complexity Representations. arXiv preprint arXiv:2306.09817.