Mobile augmented reality application for biomedical research and education

Authors

• K Zhang
• P Cruz
• M McCarthy
• D Hurt

Abstract

​Augmented reality (AR) technology combines a real-world view with computer-generated assets. AR devices range from high-end wearable headsets to smartphone-based applications that can be used with or without inexpensive stereoscopic viewer accessories. Our research and others' work has shown the potential to dramatically change biomedical research, education, and clinical practice. We implemented a mobile application for 3D visualization of biological structures in AR using the Merge CubeTM (Merge Labs, Inc.), a 2.75" foam cube. A smartphone camera detects markings on the outer surface of the cube onto which a hologram is projected, thus allowing the user to manipulate the virtual object with their hands. Our mobile app prototype uses the Merge Cube software developer kit to display 3D models of biological structures, either on the flat smartphone screen or with a wearable smartphone accessory for stereoscopic viewing. 3D manipulation of these models in AR provides a novel and convenient method for researchers and students to communicate and learn about structural and functional features of biomolecules and viruses. Our alpha version is developed for the Apple iOS platform, and we are incorporating additional biomedical structures besides influenza virus models and interactive features. Demonstrations of the app thus far have received positive feedback (especially from hundreds of users including teachers, students and researchers during 2018 US Sceince and Engineering Festival), and more extensive user testing is underway with a small number of NIAID research staff. Our intention is to extend the app for Android OS capability and to make it freely available to the public. This research is supported in part by the OSMO of NIAID.

Scientific Focus Area: Computational Biology

This page was last updated on Friday, March 26, 2021