2. The Course

In 2019, the first author joined Columbia College Chicago with a mandate to develop an Immersive Media BA and minor program. This process resulted in a series of intensive courses superseding two survey courses¹ that primarily introduced students to an overarching concept of immersive media, spanning augmented reality, mixed reality, projection mapping, and immersive theater without covering technical training or theoretical grounding. These revamped courses, as part of the degree program, were designed to shape and define the discipline of immersive media for students. In spring 2021, a pilot version of this course was launched, driven by student interest and administrative direction. Initially offered as an elective, the course is intended to form a cornerstone of Columbia College Chicago’s upper-level Immersive Media BA degree. Conducted remotely via Microsoft Teams in a synchronous modality amidst the pandemic, the course met only once in person at the end of the semester in May, following public health guidelines.

Consistent with previous discussions on IDN pedagogy, creating IDNs requires a “practice-based education” [1]. Accordingly, the course was scheduled to meet once a week for three hours to accommodate both direct instruction (lectures) and practice- based learning. This schedule was intended to meet the course’s goals :

1. Develop an understanding of the design and development techniques for VR experiences (Virtual Environment Design, Unity programming, Spatial Audio, VR UI, VR UX, and Interaction Design
2. Construct a VR experience with best practice and artist-led techniques
3. Understand IDN story structures and implement them in a VR experience
4. Articulate and receive precise feedback through usability

Given the breadth of content, effective time management is crucial. Instructional hours are carefully partitioned to meet these objectives. The course time is divided into three one-hour sections. For the initial five weeks, two hours were dedicated to lectures on IDNs in VR, with the remaining hour marked for design, coding, or development activities. The next five weeks see an equal split between lectures and studio activities. The final five weeks are primarily devoted to students’ independent work on their VR IDNs, with iterative refinements based on usability studies to create a final product.

The course covers C# programming in Unity, locomotion, spatial audio, virtual environment design and digital scenography, interaction design, VR UI and UX, usability tests and feedback, IDN story structures, and writing strategies. Critically, these topics needed to scaffold effectively over 15 weeks to enable students to produce a compelling portfolio piece [16].

2.1 Scaffolding the Modules and Learning

Effective scaffolding in curricular design and instruction is critical to student development, knowledge retention, and its practical application in the production of prototypes. Within the context of teaching an emerging medium, there are two concepts worth considering. First, simply enabling students to replicate design concepts in written or practical form does not necessarily cultivate their capacity to autonomously navigate the affordances of their medium, utilize available resources, or devise solutions to unforeseen developmental hurdles. Regurgitation, in short, fails to nurture the creative and critical engagement indispensable for generating compelling work. Lev Vygotsky’s Zone of Proximal Development (ZPD) provides a framework to understand this perspective.

Vygotsky defines the ZPD as “the distance between the actual level of development as determined by independent problem solving and the level of potential development as determined through problem-solving under adult guidance, or in collaboration with more capable peers”[17]. Given the wide range of initial skill levels among students in a course, it is necessary to regularly assess the gap between what was possible for students to achieve and what they could feasibly accomplish. The instructor has the responsibility to foster active problem-solving skills among students. This involves teaching them to read documentation, encouraging them to seek assistance when encountering difficulties, and promoting engagement with communities like Stack Overflow.

Importantly, in an emerging field problems stem from various domains such as narrative and visual design, mechanic development, programming, interface design, usability, and user experience design. These problems constantly evolve with new technologies and changes in the industry, presenting challenges for developers at all levels. For some students, these problems are well beyond their capacity to solve. The distance between the students’ “level of development” and “potential development” has to be closed through careful scaffolding.

Scaffolding should be considered “a special type of support given by a teacher to a student when performing a task that the latter might otherwise not be able to accomplish.” [18] Scaffolding mechanisms are embedded in curriculum content, module interrelationships, and through instruction. Originated by Jerome Bruner, scaffolds aid students in enhancing their problem-solving abilities. The scaffolding is executed through several steps:

1. Recruitment: The nature of VR as an exciting medium is central to recruitment efforts. Getting students signed up for this kind of challenging course involves employing materials that encourage their creative and critical curiosity to address the problems of designing and developing for an emerging medium. [20]
2. Reduction in Degrees of Freedom: Instead of beginning with the most complex version of the task, a simplified version is presented. This process enables the students to understand foundational concepts before moving on to more complex problems. Educators teaching an emerging medium should reduce their degrees of freedom. Such a reduction helps to scope the ideal results for student projects. Further, the scoping helps to reduce complexity overall and can lead to more achievable outcomes.
3. Maintenance of Direction: Keeping students on track and focused on solving the problem. In the course, this is achieved through the weekly group and one-on-one meetings with students as they encountered Students need access to tutors that can assist them as well. Regarding the case study, Columbia College Chicago employed exceptional undergraduate assistants to help their peers with programming and design challenges. These tutors were available via Microsoft Teams.
4. Marking Critical Features: Involves marking where students might improve on the work and providing guided feedback to help students achieve their ideal potential development. Marking critical features enable the instructor to motivate students to reach further and design more The instructor can point out what they are doing and how they might go farther. In the course, this occurred through graded evaluations and in one-on-one sessions between the students and the instructor. Depending on the module, more time can be given to technical challenges. For example, in the early modules, discussing design challenges and where students might improve can occur in an hour. Toward the end of a semester, this might take longer when more time is given to production and implementation.
5. Control of Frustration Levels: As expected, students will get frustrated with the issues that arise when developing an immersive experience on an emerging As an educator, it is critical to help them build a capacity to manage that frustration and stand with them in it. The students need to feel like the instructor is a capable collaborator able to help them turn their frustrations into constructive efforts. That transmutation of frustration into tangible progress is critical. Further, it helps students understand the nature of obstacles and how to approach them meaningfully.
6. Demonstration and Modeling: This is not showing students the perfect implementation or a ready-to-use model. Instead, it is about modeling for the students how to solve the problem meaningfully. Again, this approach is about building the capacity for creative problem-solving and imaginative storytelling. Accordingly, all instructions, worksheets, and other materials presented flexible and loose Students were encouraged to seek out additional implementations the instructor did not provide. This process helps the student realize their agency.

The application of scaffolding played an indispensable role in the structure and progression of the course. It allows for a tailored learning experience that considers the varied skill levels of the students and their capacity for development. Inherent in the scaffolding approach is the critical recognition that students require more than mere reproduction of theories, concepts, and technical materials. Instead, they need to cultivate a set of problem-solving skills and a creative mindset that enable them to navigate and make innovative contributions to an emerging and ever-evolving form like IDN in VR.

2.2 The Course Structure

Students iterate on their VR IDN over the semester. From day one, the expectation that they produce a complete and comprehensive IDN for VR is established. A rubric for this final project can be found in Appendix A. During the first seven weeks of the semester, students focus on building their concept, diegetic material, and a usable late- stage alpha or early-beta version of their IDN. Students must turn in this rudimentary build for usability testing and feedback for the midterm, due at the end of the first seven weeks. Over the final eight weeks, students iterate two more times before turning in their final experience with a cinematic trailer for their portfolio. The entire course structure is in Table 1 below. Longer explanations of course content and scaffolding follow.

Table 1. IDN in VR Course Schedule. Classes were broken up into lectures, instructor-led activities, and studio time. Subjects and practices would often meld from one class to the next.

Interactive Digital Narrative Overview. Early lectures on IDN survey the field’s history [20], what differentiates IDN from traditional narrative[21], dramatic agency[22], and canonical IDN works suggested by Hartmut Koenitz and Mirjam Palosaari Eladhari [4]: namely Façade [23], afternoon, a story[24], and Save the Date [25]. Further, VR IDNs from SideQuest are assigned. Marie-Laure Ryan’s work on textual architectures is also presented in Week 3 [26]. These architectures are then used in the following weeks to scaffold the production of the IDNs. With only 15 weeks, the goal was to get students building as quickly as possible. This rush to development left little time to build literacy in the young conventions of VR IDNs. This rush had consequences on concept development and implementation of more advanced interactions. To ensure practical training, each student is equipped with a VR headset, accessible throughout the course. In the case study, Oculus Quests were used for this purpose.

This rigorous schedule aims to swiftly transition students from theoretical learning to hands-on construction within the 15-week course duration. However, this accelerated pace towards content creation offers limited time to foster a nuanced understanding of the emerging conventions of VR IDNs. This constraint can impact the students’ capacity to conceptualize and implement advanced interactive elements. To counterbalance the impediments of the rigorous schedule, students had to be motivated to tackle complex problems while at the same time be given a foundation to do this work.

To those ends, scaffolding in this module involves Recruitment and Reduction in Degrees of Freedom. Recruitment entails stimulating student interest and assessing their aptitude for the forthcoming coursework. Implementing a pre-course survey can be beneficial for this purpose, allowing instructors to evaluate student interest, availability, and foundational skill levels. Such insights can inform the differentiated instruction strategies essential for effective learning within the course [27].

The Reduction in Degrees of Freedom is achieved by commencing with basic examples, helping students grasp foundational concepts. By studying established IDNs on platforms like Twine or within archives such as those run by the Electronic Literature Organization, students establish a foundation for considering more complex narrative designs in VR.

Interactive Digital Narrative Writing Strategies. In the context of writing strategies, Ryan’s story structures [28] are taught as fundamental frameworks to structure IDNs. In particular, the ‘Vector with Side Branches’ structure is introduced as a basis for designing a Spatial Storyworld Architecture [28, 29]. This approach encourages students to visualize each branch of the vector as a distinct VR environment. As Asim Hameed and Andrew Perkis discuss, from each space come moments for dramatic agency and storytelling [30].

In alignment with Colette Daiute and Hartmut Koenitz’s recommendations, students participate in character development exercises [3], which encourage them to create Non-Player Characters (NPCs) that serve as branching points for dialogue and narrative in their VR spaces. Concepts of worldbuilding are introduced along with immersion [21, 29] and Sense of Presence (SoP) [31–33]. Subsequently, students design their narrative world, curate spaces within it, and construct narrative beats aimed at enhancing and sustaining immersion and SoP.

In this context, Reduction in Degrees of Freedom manifests as the presentation of key IDN writing strategies and examples prior to students integrating their created content into a branching narrative system in Unity, the chosen game development engine for the course. Concurrently, scaffolding for Maintaining Direction during content creation is applied to ensure that students stay focused and on track, primarily through peer review. Following this creation process, students’ story worlds and characters undergo an in class peer review before being revised. These critiques happen in Week 3 during a 3-hour class period and as a homework assignment. This session invites detailed feedback on the storyworld, narrative structure, and characters. Each of these components is allocated an hour for a comprehensive review. For example, to aid students with character development, they can be given a worksheet (available in the Github Repo) that covers 38 character traits. It also includes four prompts to help them to develop their characters. Using this worksheet, students engage in multiple conversations with peers about how they could add more depth, life, and detail to the characters for their story. This process can be followed with additional worksheets for the storyworld and structure.

Additionally, During the case study, further support measures were necessitated by the disruptive effects of the COVID pandemic on student progress. These included structures to facilitate teacher check-ins during the peer review process over Microsoft Teams.

Basic Interaction and Locomotion Design. VR IDNs that achieve immersion support a SoP for interactors that enforces an affective experience [34] and provides the dramatic agency required for consistent, transparent, and meaningful interactions [21, 26, 35, 36]. As part of the course curriculum, students are introduced to the fundamentals of interaction design and programming. This includes both diegetic and non-diegetic interactions, developed and implemented via Unity tutorials. Although advanced narrative game mechanics are explored in later modules, students begin to understand the significance of interaction design at this stage, both as an element integral to the narrative [22] and a source for joy and play [37] supporting immersion. Comprehensive video tutorials for designing foundational grabbing and pointing interactions, complemented by Unity packages of assets and code, should be provided. The introduction of a third-party interaction framework can expedite the development and design process for the students.

Locomotion Design is critical for creating immersion and exploring an IDN with a Spatial Storyworld Architecture [37–40]. As locomotion in VR is also an accessibility concern, it’s imperative to consider an interactor’s natural mode of locomotion [41]. The course, therefore, teaches students to implement multiple forms of locomotion, including (1) impossible space architectures [42], (2) joystick, (3) teleportation, (4) flight, (5) climbing, (6) and vehicle or platform based [43].

At this point in the course, Vanishing Grace [44] is introduced as a relevant case study. It is a narrative puzzle game that uses spatial exploration, two modes of locomotion, good voice acting, and ludic and narratological interactions. It is also a case study in limiting the scope, standardizing interactions in the world, and designing meaningful spaces. Another example is the Last Labyrinth [70], a true VR IDN with multiple endings that utilizes puzzles and social presence to achieve immersion. Interactors communicate with Katia, who is unintelligible, to escape a labyrinth of horrific traps. The bond created with Katia is used to raise the emotional stakes and is instructive for students in producing SoP. Further, limited interactions—nodding and shaking the head, using a laser pointer—demonstrate to students that they do not need complex interaction designs to create an active and rich IDN in VR.

Scaffolding methods in this module include Marking Critical Features and Maintenance of Direction. The former entails highlighting areas of potential improvement for students, such as suggesting cleaner coding or design practices. For instance, if a student implements a basic inventory system found online that doesn’t effectively utilize Unity’s input system, an instructor or tutor can guide them towards more appropriate implementation. Maintenance of Direction is facilitated through the support of three peer tutors accessible via Microsoft Teams or Zoom. Having these additional resources proved critical to the success of the course, given that addressing every concern promptly would have been challenging for a single instructor.

Virtual Environment Design and Digital Scenography. The prescribed story architecture of the course necessitates a comprehensive understanding of Virtual Environment (VE) Design and Digital Scenography. The goal is to foster spatial immersion by focusing on narrative game spaces [37]. The course material for level design draws inspiration from the works of professionals such as Miriam Bellard from Rockstar North [45] and other established creators [46–48]. As there exists a significant semantic overlap between narrative, experience, and level design in games, students are encouraged to consolidate these concepts into a form of digital scenography. This approach leverages the design and spatial composition to narrate stories and support immersion [30, 49]. These practices include how to use light [50], compose spatial sound [51], and use color to direct user attention [52, 53]. Both in-class and homework activities are centered around creating these immersive environments, covering both technical processes and conceptual work.

The Book of Isabel [54] is a proposed experience for this module. This exploratory walking simulator guides the interactor through a traumatized girl’s mind. Interactors walk through memories and hear Isabel discuss the event and her recovery. With limited interactions but an emotional story, the experience is a good template for students with limited programming experience. Given that it was created by an undergraduate from the Academy of Art and Design, St. Joost, it presents a relatable and motivational example for students. A Fisherman’s Tale [68] is another proposed case study. This VR experience is a puzzle game that plays with perception. It uses recursion and chains of interactions to create a world wherein the interactor is controlling an avatar, that is controlling another avatar, and so on. It is a case study of using non-Euclidean space in VR to create compelling interaction spaces. For students, it helps to inspire creative ideas outside of their familiar game worlds and their natural reactions.

In terms of scaffolding, this module integrates several steps: Maintenance of Direction, Marking Critical Features, Control of Frustration Levels, and Demonstration and Modelling. Maintenance of Direction becomes vital as students start to understand the significant time and focus required to build virtual environments. As students progress, Control of Frustration Levels also becomes crucial, especially when they grapple with project scopes, encounter obstacles, and balance their outside commitments.

Aiding students in controlling frustration sometimes necessitates Reducing Degrees of Freedom, slowly scaffolding towards increased complexity. For instance, if a student is struggling with implementing a VR grabbing mechanic but understands ray casting for object selection, the instructor can first help them implement the latter, enabling a sense of progress before moving onto the more complex task. Preparing students for complexity is also addressed through Marking Critical Features; for example, this is when an instructor might guide a student who has developed repetitive bird animations to Craig Reynold’s Boids, a flocking and swarming algorithm that yields natural behaviors, thereby elevating the authenticity of their virtual environment. These methods of Demonstration and Modelling instruct students in problem-solving strategies for their design and development challenges.

Rudimentary Artificial Intelligence and Narrative Game Mechanics. In VR, worlds enforce immersion and a sense of social presence when NPCs exist [55]. The behaviors of NPCs, as well as animals, machines, objects, and more, within this world not only enhance narrative immersion and social presence but also lend an embodied scale [56] and scenographic energy [57] and help direct their attention [58]. Impressing upon interactors that there is emergent behavior in the space enforces the narrative immersion [29, 59].

Narrative game mechanics are introduced. As defined by Teun Dubbelman, “Narrative game mechanics invite agents, including the player, to perform actions that support the construction of engaging stories and fictional worlds in the embodied mind of the player.” [19] These mechanics go beyond the foundational interactions discussed earlier in the semester. Initial mechanics focused on user interface interactions or non- diegetic interactions. Narrative game mechanics scaffold from those fundamental interactions and may support feelings of embodiment in interactors. Helping students conceptualize and implement these mechanics was a crucial challenge.

Considering the distinctiveness of each IDN, materials and resources for these mechanics are created through personalized instruction and peer collaboration that were intentionally designed into a process for scaffolding student capacity and motivation for storytelling in VR. In one-on-one instruction, students discuss the narrative game mechanics they are planning or have implemented. The instructor can then Mark Critical Features when reviewing these mechanics or through Demonstration and Modeling provide additional resources to facilitate implementation. For instance, the instructor facilitates the transition between this and the previous module, i.e., from Virtual Environment Design and Digital Scenography to Rudimentary Artificial Intelligence and Narrative Game Mechanics, by demonstrating how simple wandering NPCs bring more life to the environments they designed. As an example of Marking Critical Features, the module introduces Finite State Machines (FSM) to control various AI agent behaviors—a more complex concept than a simple wandering algorithm.

Simultaneously, student groups are encouraged to review and support each other in overcoming challenges. The peer group discussions and reviews help Maintenance of Direction. This approach, though labor-intensive for the instructor and tutors, allows for patterns to emerge, leading to the development of templates for common mechanistic implementations. For instance, if multiple students require a dialog option resulting in an item addition to the inventory that will eventually allow them to pass into a forbidden area, the instructor can construct this as a template. By carrying over these templates of narrative game mechanics across different course iterations, students can focus their efforts more effectively and save instructional time.

One excellent example to show during this module is Moss [60], a third person VR adventure experience that upsets expectations for VR narratives. The interactor plays a god-like character that directs Quill, a young mouse. The spaces are lush, but more importantly, Moss is a case study on building a social sense of presence for the interactor with a character. When the interactor is inactive, Quill will look at them and respond to their inactivity. This interaction helps to teach the importance of character development and social presence and their impacts on immersion. Another excellent example of this module is Shadow Point [69], a VR puzzle experience wherein diegetic content helps interactors solve puzzles to find a missing woman. Each world provides a new kind of interaction to script the interactor for future scenarios appropriately. It is a good case study on connecting ludic elements and creating narrative game mechanics.

VR User Interface and User Experience Considerations. The curriculum encourages students to understand and adopt best practices for UI and UX. These aspects are taught simultaneously with students learning from Oculus Quest [35] and LiminalVR [61]. LiminalVR’s collected research on the psychological impacts of immersive design choices is valuable for IDN work. As in previous work on IDN curricula, engagement with the psychological impacts of interactions is required [3]. The research spans topics like the effects of motion, color, lighting, music, sound, interactivity, ludic interactions, flow, and cognitive load. Familiarity with these concepts can enable students to achieve immersion and a sense of presence (SoP).

Students are encouraged to develop as many interactive narrative game mechanics as possible as an effective UX is necessary for an IDN [11, 62]. For example, one student created a teleportation interaction that looked like a rope that was, in some way, connected to the character’s story and the world. This expectation is meant to encourage UI and UX that achieve Jay Bolter’s transparency in service of immersion [63–65]. In another instance, one student’s story involved a bully being turned into a gecko who then must converse with bugs. To begin the game, the interactor must open a box. Afteropening the box, insects scurry around inside, their movement encouraging the user to lean in. When the user leans in, an interaction triggers a scene change that starts the narrative.

The course also instructs students on how to conduct the ‘Think Aloud’ method [66] for their usability studies. During the pandemic, students designed their VR experiences in isolation, which posed challenges for usability studies. Consequently, they were not introduced to quantitative methods due to time and scope limitations, despite their considerable value. Regardless of whether the course is conducted remotely or in- person, students are advised to record their playthroughs while verbally sharing their thoughts and interactions. Their peers would then convert these insights into actionable steps for subsequent iterations. Students engaged in this peer review process three times over the course’s final three weeks, allowing them to receive feedback from the entire class while progressively refining their narrative.

Under ideal circumstances (non-pandemic), students would apply this method with multiple peers both during and outside class time, thereby ensuring a robust use of Maintenance of Direction scaffolding. In a classroom setting, it’s easier for the instructor to observe the process and provide immediate constructive feedback. Moreover, students should be introduced to different usability tests such as the user study prototype pair protocol discussed by Koenitz and colleagues [1]. This method could have significantly amplified the amount of feedback received and facilitated earlier testing if all students were in a single location.

2.1 Financial, Logistical, and Technical Challenges in Teaching

There are several challenges associated with executing the proposed curriculum, arising from the rapidly evolving IDN and Immersive Media fields and varying student literacy levels in these domains. To build this effective learning environment, educators must strike a balance of technological, pedagogical, content and contextual knowledge. This blend, often referred to as TPACK (Technological Pedagogical Content Knowledge) in learning theory[67, 68], must account for the unique institutional and circumstantial contexts, such as a pandemic, university, state, or federal policies governing student technology use, financial challenges, and socio-techno-cultural contexts.

Some students may not have had prior exposure to VR, necessitating a steep learning curve to comprehend storytelling within this medium. The previous section suggested several immersive experiences, which, if required for purchase, would cost students $115 (as of writing in 2022). Solutions to alleviate this financial burden and increase accessibility could include renting or sharing VR experiences, reintroducing free demos, or establishing educational licenses for experiences that allow multiple students to engage concurrently. Collaborative efforts between university libraries or departments and the industry could facilitate such solutions.

Determining incoming students’ technical capabilities, visual or digital design skills, and prior knowledge in IDN, VR, or game design[69, 70] presents a significant challenge, especially in an emerging field. Self-reporting through pre-course surveys can be misleading [71], and students may come from different traditions entirely if the course is an elective. Balancing the educational material’s rigor for advanced students while ensuring accessibility for beginners, particularly in areas like programming and development engines like Unity, is complex. However, having teaching assistants or tutors can help come up with diverse learning materials that meet learners where they are at.

Access to hardware and experiences is an issue. The Oculus Quest was used in the case study and may be an affordable option. However, the recent announcement that ads would become part of VR experiences may not make the Quest acceptable [72]. Further, students may, and did, resist being forced into the Facebook ecosystem when the course ran. Despite the ability to create independent Meta accounts, students have expressed hesitation about contributing data to a corporation perceived as problematic. This sentiment from students raises significant institutional questions about which headsets and ecosystems a university adopts. One work around might be creating general accounts unlinked from students’ identification information. However, students may then lose access to the work or experiences they purchase. Alternatively, institutional accounts managed by university libraries could be a more comfortable option for students and allow for renting VR experiences. Policies need to be developed for the borrowing, return, and maintenance of VR headsets for the students to use. This may mean wiping them off data between uses to avoid privacy issues.

Physical space can also pose a challenge. While some students can create VR experiences in large apartments, others may only have access to a small bedroom or dorm. To ensure equity, a common space on campus should be made available for students to experiment, supplemented with a VR emulator accessible to all [73]. The emulator will speed development for all students regardless of the size of their physical space. The case study was conducted during COVID, adding its own constraints. In a non-pandemic scenario, VR headsets should be provided for students for use in a common open space that enables multiple students to work concurrently.

The course modality also presents challenges. While a remote synchronous modality, as used in the case study, allows for lectures, discussions, and individual check-ins, it makes immediate feedback and project management difficult. For the case study, this was all done over Microsoft Teams. Teaching in a classroom allows students and instructors to use a VR headset in a shared space, which can enhance communication around embodiment and immersion concepts. In contrast, teleconference platforms flatten the spatial nature of VR and make it challenging to communicate these concepts effectively. Students miss valuable insights when they can only see a screen recording of a VR experience rather than the entirety of an interactor’s body and the physical space they are occupying. Therefore, future iterations of this course should ideally be conducted in-person.