Presentation at ETUG: Co-creating XR Application for Education

What is ETUG?

Educational Technology Users Group (ETUG) is a community of BC post-secondary educators focused on the ways in which learning and teaching can be enhanced through technology.

What is this presentation about? 

In the ETUG Fall 2018 Workshop, I proposed to present. And here is the abstract:

Most educators get to access AR/VR/MR applications when they are well-developed and ready to use, but they are missing in the design and development process. To most people, the design and development process remains mysterious, but educators need to be involved in the process and become co-creators. Sharing the process and challenges will be an important first step.

What was presented?

Developing HoloLens Application for Aircraft Maintenance Engineers

Just in case you don’t know, our project is basically to translate an existing desktop Helicopter Rotor Head application into Microsoft Hololens application. In addition, we are adding voice control and multiple user network.

(Image Credit: John Bondoc, UX Designer)

Sounds interesting? It’s actually quite scary because none of our team members had experience developing HoloLens application previously. So we learn as we do it.

We just finished Sprint #6 and entered into Sprint #7.

(Credit: Junsong Zhang, Project Manager)

In Sprint #5, we created a prototype based on the concept art below and started adding voice control into the application. The main changes were: 1) we added a voice control command board, and 2) the controls were moved below the rotor.

(Image Credit: John Bondoc, UX/UI Designer)

However, in Sprint #6, we tested the prototype and found that the main problems are

  • Difficulties with collective/cyclic controls.
  • Difficulties to see the effects such as airflow & swashplate movement.
  • Difficulties in looking at the rotor while moving the cyclic/collective controls.
  • Slow response with voice control.
  • Confusion about voice control menu.
  • Insensitivity of voice control in noisy environment.
  • There are no indications when “make bigger/smaller” voice controls hit the limits.
  • Not enough training/instructions with regard to how to use HoloLens.
  • Users tend to take it as a 2D object instead of 3D.

Considering scope, we decided to work on the ones that are critical to the functionality of this application:

  • Improve cyclic function
  • Resize/reposition cyclic & collective controls
  • Resize the whole field of view so users don’t need to move their heads too much at the beginning.
  • Change how the model scale up and down: keep the rotor in the background while it grows bigger or smaller instead of jumping to the front.
  • Redesign the voice control menu: instead of a command board, the new voice control menu will be interactions that give users instructions when they gaze/hover over the buttons.

(Credit: the entire team)

Based on that, we came up with a new sketch that reflects that our new interface and interaction. The main changes are the positioning of controls and voice control menu, as well as how the interactions work. We’ll have to prototype and test it. 

(Image Credit: John Bondoc, UX Designer)

From this week on, we will spend more time developing a multiple user network. The idea is to enable instructors to broadcast their views and modifications in the application to students in real-time.

Learning Design Theories and Practices for PMs, Designers and Developers

The beauty of our team is that we are a team of 6 with diverse backgrounds and skillsets. That means we are able to, and we should, create as many learning opportunities for each other as possible in various fields including but not limited to project management, strategic communication, stakeholder management, marketing, programming, 2D/3D design, UX/UI design, game design, and learning design.

In this post, I’ll put together a crash course on learning design theories and practices for my team that is about to design and prototype a functioning virtual reality training simulation in 12 weeks.

Reviewing what I learned during my master program in Education, I selected some theories, principles, and practices for the team. They are important and relevant to this project yet not difficult to understand, and some of the theories and principles could be seen as common sense for people who are naturally empathetic.

So here is the list:

  • Cognitive Load Theory
  • Principles of Multimedia Learning
  • First Principles of Instruction
  • 4C/ID Model
  • Worked-out Examples
  • Principles of Adult Learning

As we are designing a VR simulation, another important task for me throughout this project is to contextualize these theories and discover how these principles may or may not apply in VR design.


Cognitive Load Theory

Why? Cognitive load theory provides a high-level understanding regarding how human brains process information and is the foundation for multimedia learning principles. 

Key Concepts

  • Sensor Memory
  • Short-term Memory
  • Long-term Memory
  • Chunk
  • Extraneous Load
  • Intrinsic Load
  • Germane Load

Key Messages

  • Do not OVERLOAD users because our cognitive capacity is limited.
  • However, how much information is too much for users depends on individuals’ mental model (schema), which is why we need to research and differentiate the level of our users/learners.

Selected Resources


Principles of Multimedia Learning

Why? Multimedia learning principles are evidence-based guidelines and practices that aim to decrease extraneous load, manage intrinsic load and maximize germane load. 

Mayer’s 12 Multimedia Principles:

  • Coherence Principle 
  • Signaling Principle
  • Redundancy Principle 
  • Spatial Contiguity Principle
  • Temporal Contiguity Principle
  • Segmenting Principle
  • Pre-training Principle
  • Modality Principle
  • Multimedia Principle
  • Personalization Principle
  • Voice Principle
  • Image Principle

In addition, Merriënboer offers three design principles and strategies:

  • Variability principle
  • Contextual interference principle
  • Self-explanation principle

Selected Resources


First Principles of Instruction

Why? First Principles of Instruction is basically a synthesis of all major instructional theories. It is easy to understand and probably the best starting point to learn about learning design. 

Selected Resources


The 4C/ID model

Why? The 4C/ID model stands for four components instructional design model. It is a problem-based approach that is widely used in technical/medical training.

The four components in 4C/ID model are:

  • Learning tasks
  • Supportive information
  • JIT information
  • Part-task practice

Selected Resources:


Worked-Out Examples

Why? Worked-out Example is a great approach to break down learning steps and show learners what to do. Renkl points out that learners gain a deeper understanding when they receive well-designed worked-out examples at the beginning of cognitive skills acquisition.

Selected Resources:


Principles of Adult Learning

Why? Because teaching adults are fundamentally different from teaching kids or teenagers. Understanding these principles will inform the high-level learning design for adults.

Selected Resources:


Reference

Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and Development, 50(3), 43-59. doi:10.1007/BF02505024

Mayer, R. E. (2002). Cognitive theory and the design of multimedia instruction: An example of the Two-Way street between cognition and instruction. New Directions for Teaching and Learning, 2002(89), 55-71. doi:10.1002/tl.47

Mayer, R. E. (2008). Applying the science of learning: Evidence-based principles for the design of multimedia instruction. The American Psychologist, 63(8), 760-769. doi:10.1037/0003-066X.63.8.760

Renkl, A. (2005). The worked-out-example principle in multimedia learning. In Mayer (Ed.) The Cambridge handbook of multimedia learning, (pp. 229-245). New York; Cambridge, U.K: Cambridge University Press.

Roy, M., & Chi, M. T. (2005). The self-explanation principle in multimedia learning. In Mayer (Ed.) The Cambridge handbook of multimedia learning, (pp. 271-286). New York; Cambridge, U.K: Cambridge University Press.

van Merriënboer, J. J. G., Clark, R. E., & de Croock, M. B. M. (2002). Blueprints for complex learning: The 4C/ID-model. Educational Technology Research and Development, 50(2), 39-64. doi:10.1007/BF02504993

van Merriënboer, J. J. G., & Sweller, J. (2010). Cognitive load theory in health professional education: Design principles and strategies. Medical Education, 44(1), 85-93. doi:10.1111/j.1365-2923.2009.03498.x