It’s Now: Designing and Using Immersive Simulations for Learning

Posted on July 21, 2021 by junsongz

Virtual reality (VR) or immersive VR simulations for learning is no longer something to be imagined or planned in the future. If you have not done a project like this yet, the best time to start is now.

About two years ago, I had the chance to present “Virtual Reality for Public Safety Training” at JIBC Open Symposium. After introducing relevant pedagogical significance, examples, research on effectiveness, and future opportunities, I landed on a question for the audience, “What would be the problem you want to solve with emerging technology like VR?”

The responses were fueled with enthusiasm and revealed a wide range of applications and practices that JIBC could benefit from. Just to name a few:

“simulate a patrolling scenario in downtown eastside”
“access correction centers and walk around the buildings”
“build empathy by living and acting like someone else”

As we can see, common to these scenarios are things that we feel difficult to design or simulate effectively in the classroom. Bailenson (2019) pointed out that high-fidelity 3D environments like virtual reality have the capability to simulate scenarios that are:

Impossible: you cannot change skin colour easily, but in VR you can inhabit avatars with different skin colours with profound results.
Expensive: you cannot easily fly your whole school to Machu Picchu.
Dangerous: you would not want to want to train emergency landings by crashing real airplanes.
Counterproductive: you should not cut down an entire forest to instruct on the problems of deforestation.

The primary differentiator for high-fidelity immersive simulation technologies is that it creates a strong sense of presence and enables embodiment.

The concept of Presence has a long history in psychology, Witmer and Singer (1998) defined it as “the subjective experience of being in one place or environment even when physically situated in another”.
Embodiment, on the other hand, means “our process of learning and understanding are shaped by the actions by our body, such as gestures”. (Johnson-Glenberg, 2018)

While presence manipulates or alters how our brains perceive the world around us, embodiment allows us to physically rewire our learning from a behavioural level. Scholars suggest that well-designed 3D simulations address some critical pieces in learning from a pedagogical perspective. Dalgarno and Lee (2010) listed five major affordances of 3D virtual learning environments:

Spatial knowledge representation: facilitate learning tasks that lead to the development of enhanced spatial knowledge representation of the explored domain.
Experiential learning: facilitate experiential learning tasks that would be impractical or impossible to undertake in the real world
Motivation and engagement: facilitate learning tasks that lead to increased intrinsic motivation and engagement
Improved transfer of knowledge and skills: facilitate learning tasks that lead to improved transfer of knowledge and skills to real situations through the contextualization of learning.
Collaborative learning: facilitate tasks that lead to richer and/or more effective collaborative learning than is possible with 2‐D alternatives.

Learning these theories provided us with a foundation of how we may move forward with high-fidelity simulations. But designing and customizing simulations for specialized areas in public safety remains a challenge as it requires a wide range of expertise such as content knowledge, programming, 2D/3D modelling, animation, user experience design, and user interface design. A collaborative, interdisciplinary team is the key to the development of accessible and usable immersive virtual learning environments.

Fortunately, over the last two years, we were able to identify some internal partners at JIBC who are interested in developing simulations, and also had the opportunity to collaborate with a few highly skilled teams from the Centre for Digital Media.

Here are some video demonstrations for these award-winning projects:

Produced by LifesaVR team at the Centre for Digital Media

Produced by FlashoVR team at the Centre for Digital Media

https://youtu.be/_DvdOPLUGLg

Produced by Team Caramel at the Centre for Digital Media

Reference

Bailenson, J. (2019). Experience on demand: What virtual reality is, how it works, and what it can do. W. W. Norton & Company.

Dalgarno, B., & Lee, M. J. (2010). What are the learning affordances of 3‐D virtual environments?. British Journal of Educational Technology, 41(1) https://onlinelibrary.wiley.com/doi/full/10.1111/j.1467-8535.2009.01038.x

Johnson Glenberg, M. (2018). Immersive VR and Education: Embodied Design Principles that Include Gesture and Hand Controls. Frontiers in Robotics and AI, 81. https://www.frontiersin.org/articles/10.3389/frobt.2018.00081/full

Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence, 7(3), 225-240.

LifesaVR: Immersive Learning Experience for Paramedics

Posted on December 18, 2020 by junsongz

Looking into the Context and the Problem

Shortly after I joined one of the curriculum committees at the Justice Institute of British Columbia, I learned that the paramedicine program has a 100-page practical simulation book, detailing how instructors should describe each scenario for simulation practice. The students, wearing their uniforms, carrying emergency cases, often stand in the middle of the classroom and try to imagine the physical environments where they are supposed to save lives.

In school, students rely on verbal explanations from the instructors to assess hazards and determine their next steps, whereas they are expected to observe the environment, identify risks, and save lives independently in the field. It is clear that there is a gap between what is learned in the classroom and what is required in the field.

As an instructional designer, I hope to fill the gap by designing and developing user-centred immersive experiences that could potentially shift the way we learn and teach in formal and informal environments.

Setting up Achievable Goals

Working with a team from the Centre for Digital Media, our goal is to develop a working VR prototype that simulates immersive learning environments for paramedics. The prototype will provide both experiential and reflective learning opportunities for trainees to apply critical assessment and decision-making skills, leading to better response when real events occur.

In addition to the realism of the learning environment, our design includes: 1) functionalities to track, analyze, and review how trainees made their decisions during the simulation; 2) non-intrusive user interface and exemplary user experience for reflective learning; 3) and a final package that can be easily installed and used in different physical settings.

Building, Testing, and Iterating the Prototype

Through iterative design, rapid prototyping, and user testing, we created a simulation in which a young adult is struck by a vehicle and becomes unresponsive. In the simulation, the patient is found lying supine on the pavement, with obvious bleeding; there are broken glasses on the ground near the patient, and the gas is leaking; police have stopped the traffic and firefighters are at the scene to assist.

Students are required to identify all the hazards, and then use three dressings, in the correct order, from the emergency case to stop the bleeding. In order to analyze students’ decision-making processes, we also designed a dashboard that collects two sets of data: attention and reaction time. Attention, tracked through head movement, measures how long students look at an object, while reaction time measures how long it takes a student to complete a task. The dashboard is cloud-based so instructors can read the analysis and provide feedback from anywhere.

Presenting a Mid-Project Video Demonstration

Halfway through the project, we were asked to present this project to JIBC donors, faculty, and staff. Although the final prototype has evolved from this iteration, this video provides a succinct introduction to our design. I personally adore this video for its energy and simplicity.

Designing and Redesigning the Dashboard

The team conceptualized, prototyped, and iterated the dashboard in two or three weeks. Click the image to view the final interface of the dashboard.

Installing the Final Prototype and Putting on the Backpack Devices

Walking Through the Final Prototype

And Finally Meeting the Team

Centre for Digital Media

Eris Lam, Developer
Kyle Liu, 3D Artist | Unity programmer
Julia Read, Product Manager | UI Lead
Farbod Tabaei, Concept Artist | Game Designer
Sooq Won, UX Researcher | Graphic Designer
Yuan Zhang, 3D Artist | Game Designer
Robyn Sussel, Faculty Advisor

Justice Institute of British Columbia

Junsong Zhang, Instructional Designer, CTLI
Robert Walker, Director, CTLI
Svetlana Larson, Regional Training Coordinator, PCP
Eric McConaghy, Regional Training Coordinator, PCP

And a big thank you to people who participated in the user testings and provided valuable feedback.

Presentation at ETUG: Co-creating XR Application for Education

Posted on November 9, 2018 by junsongz

What is ETUG?

Educational Technology Users Group (ETUG) is a community of BC post-secondary educators focused on the ways in which learning and teaching can be enhanced through technology.

What is this presentation about?

In the ETUG Fall 2018 Workshop, I proposed to present. And here is the abstract:

Most educators get to access AR/VR/MR applications when they are well-developed and ready to use, but they are missing in the design and development process. To most people, the design and development process remains mysterious, but educators need to be involved in the process and become co-creators. Sharing the process and challenges will be an important first step.

What was presented?

Co-creating XR for Education: Skills, Process, challenges from Junsong Zhang

Getting Ready for a Virtual Reality Training Simulation Project

Posted on January 6, 2018 by junsongz

First of all, yah!! Why? Because I get to work on a Virtual Reality training simulation project. What could get better than this?

Super excited as this is totally the area I want to explore: learning design with cutting-edge technology. Having the experience working directly with doctors to design next-generation learning experience is going to be extremely valuable too.

Plus the team I am with looks awesome! We are starting to build the team and figure out who would fit what role(s) within Agile/Scrum method.