ZapBox Mixed Reality Kit: An Emerging Learning Tool

By Clement Abai

Background

I am obsessed with the idea of merging real and virtual worlds, which is often referred to as mixed reality. There are different tools out there that can be used to create mixed reality and each year they are becoming easier to use, especially for those who may not be expert computer coders. This of course means that the possibility is becoming greater for creative integration of these tools into teaching and learning practices. My new tool of interest is ZapBox. In this blog I’ll share a bit about what Zapbox is, and why I think it is an exciting emerging technology that should be considered for the ETC Research Lab.

What is ZapBox? 

Developed by Zappar,  ZapBox Mixed Reality (MR) kit allows developers to design, build and create 3D experiences that can be combined with the physical components such as your workspace with advanced software to provide immersive MR experiences through the use of mobile technologies such as smartphones (www.zappar.com, 2021). This implies that ZapBox MR can be used in an educational setting to create an engaging learning environment that includes the physical and virtual world. Therefore by overlaying virtual objects on the physical environment, it enables learners to interact with each other in the same physical learning space with the virtual objects (Billinghurst, M., & Kato, H. 1999). The interaction involves learners seeing each other in the space, collaborating, discussing, sharing ideas while engaging in the learning process (www.zappar.com, 2021). Below are examples of MR models or applications where virtual objects are integrated into the physical world. Figure 1 is a virtual model of the habitat and nesting ground for the sea turtles and Figure 2 is a 3D model of a turtle (www.zappar.com, 2021). 

(Image Source: Zappar.com) Figure 1: Habitat and Nesting.

(Image Source: Zappar.com) Figure 2: Sea Turtle.

This overlaying of the sea turtle habitat and nesting ground on a table gives learners an engaging learning environment where they can learn about the sea turtles and their habitat and behavior. In doing so they will be able to interact with each other and engage in the learning process.

Similarly, another MR learning tool that has been gaining popularity in the medical field is the Microsoft Hololens. A study by Jong-Chih et al. (2019) examined the use of Microsoft HoloLens MR head mouth and Intel RealSense sensor for medical use. Currently, image-guided surgery (IGS) technology is used to check and verify the progress of the surgeries. While the surgeon checks the IGS display, their attention is diverted from the patient occasionally. With mixed reality, the patient’s medical data can be displayed on the physical surface of the patient while the surgeon performs surgery without diverting their attention. Transformation mapping is used to handle noise and outliers’ problems or medical data. The transform function is monitored by the Denoised-Resampled-Weighted-and-Perturbed- Iterative Closest Points (DRWP-ICP) algorithm. The DRWP-ICP denoises and removes outliers and aligns the pre-operative medical image data points to the patient’s physical surface position. The results are then displayed on the patient’s body using Microsoft HoloLens display system. Below are holograms of human body parts (virtual human body parts) with doctors performing an ultrasound (Figure 3) and a display of the human circulatory system (Figure 4) through use of Microsoft Hololens (www.microsoft.com, 2021).

(Image Source: Microsoft) Figure 3. Ultrasound.

(Image Source: Microsoft) Figure 4. Human Circulatory System.

These studies demonstrate how MR can be used in educational fields, medical fields and various fields. In education, displaying virtual objects in a physical environment enables group collaboration, discussion and promotes engagement during the learning process. In the medical field, the use of MR during surgeries eliminates errors and provides medical data of patients to doctors efficiently and accurately during surgical procedures.  Therefore, with MR gaining momentum in education and other fields,emerging technologies such as ZapBox MR kit have the capability to enhance learning by creating a collaborative and engaging learning environment by combining the physical and the digital world should be a learning tool schools (K-12), educators and teachers should invest in in order to create fun and engaging contents for their students.

Below is a table of list of MR technologies, information of their developer and cost. 

References

Billinghurst, M., & Kato, H. (1999, March). Collaborative mixed reality. In Proceedings of the First International Symposium on Mixed Reality, (pp. 261-284).

Jong-Chih, C., Yao-Ren, T., Chieh-Tsai, W., & Jiann-Der, L. (2019). HoloLens-Based AR System with a Robust Point Set Registration Algorithm. Sensors, 19(16), 3555. 

Microsoft (2021). https://www.microsoft.com/en-us/hololens

Zappar (2021). https://www.zappar.com/zapbox