- Annex Detail -

IVLA PRESENTATION


I created these slides for the 2019 Conference of the International Visual Literacy Association (IVLA) in Leuven, Belgium. The talk is titled "Designing A Virtual Reality Teaching And Learning Environment For Chemistry." See also the Virtual Reality Layout and Virtual Reality Controls Case Studies.

T.M.Wilcox talks about crafting his first prototype of a virtual reality teaching and learning environment. Mr. Wilcox has 30+ years experience in computer game design, software development, and 3D concept visualization.

The Unity game engine has a component called a Skybox that takes six images to form a cube. Unity transforms these into a spherical environment map surrounding a virtual world. Like a stage backdrop, Skyboxes become a stylistic theme for environments in a virtual reality experience.

I use existing items from different sources to speed up the prototyping process. These island mountains are 3D geometric mesh objects from another project. The clouds come from a panoramic photo mural. The rippling water comes from a procedural wave generator. Together they make an inspiring environment for learners.

Walking on water is interesting but distracting, so I quickly made a sandy ground plane using a tiled texture map. Designing a virtual reality environment is about trying things and taking inspiration from the result. This desert-like world gave me an idea.

I was thinking about sandboxes as a basic feature of an interface design. To get started, I took out my sketchbook. A playful array of geometric sandboxes seemed like a helpful navigation aid. I created a simple 3D model of shallow boxes connected in a three-by-three grid.

Standing on the ground in VR, I couldn’t see the grid. What I needed were markers at the corners of each sandbox. For now, developing virtual reality means repeatedly putting on and taking off a headset to check your work.

Palm trees come in a Unity library of ready-made game objects. I modified their shadows adding definition to the sandbox walkways and ramps. The virtual world was rapidly taking shape as my sketchbook filled with notes and drawings of interface concepts.

What I needed was an easy-to-learn control interface for non-gamers. Driving over yellow pads became my standard event trigger for interface audio, graphics, and animation. The only input device for virtual reality in 2015 was an XBox-type gamepad controller. The default navigation method in the Oculus SDK supported walking or driving.

It was time to add subject matter content to the scene. Teaching basic chemistry was the objective. I discovered an on-line public resource of 3D chemistry models. Circular yellow pads became event triggers surrounding the models. From an interaction standpoint, there wasn’t much going on. I was beginning to doubt my design.

The VR experience was intense. My collaborators were chemistry professors who had never seen anything in virtual reality. They were astonished. A professor agreed to become a video hologram describing ball-and-stick models and other objects. Instinct told me I needed even more than that.

The environment was distracting, and it took too long to navigate. To test an idea, I built a fence around a single learning object demonstrating close packed spheres. It wasn’t perfect, but it helped me to see things differently. A theme park seemed okay for casual learning. For deeper learning, I wanted something else.

I went back to my skybox environment and created a new ground plane with a group of seven hexagons referencing a six sided carbon molecule. I saw the central area as a formal instruction zone. The surrounding panels would hold props for learning objects that could move on and off stage as required.

Navigation controls were transformed as well. Flying is easy to implement in virtual reality, and it offers students unlimited observation viewpoints. To make it work, I needed a mental model for newcomers and non-gamers. So I programmed the gamepad making it feel like you were gently flying a hot air balloon.

I added waypoint navigation to the gamepad affording instant teleportation to preprogrammed locations. Instant location changes in VR aren’t as disorienting as once predicted. Having some continuity in the scene is sufficient for most people. Careful design and testing is critical.

The Close Packing learning object is also controlled with a gamepad. Students can start, stop, and reverse the 3D animated spheres while watching them from any angle. Close packing is a fundamental concept in college chemistry. In all, sixteen learning objects were planned for a complete product, each one hosted by a chemistry professor avatar.

By 2017, Oculus had shipped their HID compliant, dual Touch controllers, allowing event triggering and object repositioning using hand movements. In 2020, Oculus expects hand tracking technology will begin replacing physical controllers, making VR interaction much more intuitive.

Inorganic chemistry textbooks were a primary source of information for this project. Page after page had two dimensional symbols representing complex 3D structures. Shadows underneath the VR models looked remarkably similar. Unexpectedly, they help make the connection between enactive and symbolic representations of molecules.

I named the prototype software “ChemSim-VR” (CHEMistry SIMulator for Virtual Reality). It was a part-time project for two years, perhaps 600 hours total. Chemistry students with low spatial reasoning ability benefited most from ChemSim-VR. It’s likely all students benefit from VR visualizations of complex STEM concepts.

The ChemSim-VR Prototype is an early example of a virtual reality teaching and learning environment developed by a game designer for higher education. Lessons learned inform current research and development of virtual reality at the Concept Visualization Center in the College of Education at James Madison University.