Assistant Professor of Practice in Emerging Media Arts Anna Henson taught a special topics course this semester titled “Creating Augmented Worlds,” to explore virtual and mixed reality and explore how people can be together, even if they’re far apart.
“Virtual reality has been asking that question for a while, and there are many tools,” Henson said. “Telepresence [such as a Zoom meeting] is one method for being together. But Zoom class can sometimes lack the embodied connection, the feeling of being in the same space together. This presented a really great opportunity to see how being together in a virtual space actually contributed to our learning experiences. And for the students themselves who may end up being designers or creators or technologists in this area to try to unpack from their own actual experience what virtual reality might be in the future and how they might want to design or participate in that.”
Henson began teaching in the Johnny Carson Center for Emerging Media Arts last fall. She has had a long career as an educator, creative producer for tech startups, VR companies and experiential agencies, as well as a decade-plus career as a projection and media designer for Broadway, Off-Broadway and regional theatres.
“The industry has been asking the same question. Not only the tech industry, but other industries like theater and live performance, which has had to shut down physically,” she said. “How can we experience live performances? I have a big background in theatre myself as a projection and media designer. I’ve been talking with a lot of my professional colleagues and working on shared theatrical experiences in virtual space, also, so I’m bringing in some of that dialogue to our class as well.”
The course offers a “big picture” introduction to the possibilities offered by virtual reality and immersive 3D experiences and discusses the broader universe of augmented and mixed reality. Students began with individual prototyping assignments and worldbuilding in Unity, a game engine. Students also explored group activities in social VR, such as VRChat, Rec Room and Mozilla Hubs. The class culminated with final group projects, splitting into three groups of three students each, to create a world and an avatar for VRChat.
“I’m really wanting to expose the students to a wide variety of these platforms so they can start to analyze them themselves and experience their avatar or their embodiment across different platforms to start to unpack the design strategies, the technical limitations and the possibilities that are there,” Henson said.
In addition, students completed short readings, podcasts, videos and discussion posts throughout the course , and they kept an experience diary.
“I’m really a big proponent of heavy documentation of your process and your work, partly because your portfolio will benefit from it,” Henson said. “If you continuously document the whole time, you will have a lot of material, whereas some of these projects can be very ephemeral. The software that you build now probably won’t work in a few years, so it’s not like you can go back and re-experience it at any time. So in having students document their process as well as polished projects, I really want them to get in the habit of documenting early and often. In addition to the visual or interactive work they create, I’m also having the students write in a diaristic manner about their process, what I’m calling ‘experience diaries.’ How do you feel about learning in a virtual space while we’re remote? How is this changing or impacting your learning experience to have this embodied platform to play with and to work with and to be talking about virtual space, while we are talking to each other in virtual space?”
Henson said the voices of her students matter in this growing community and industry of virtual and augmented reality.
“By having my students record their thoughts and processes through experience diaries and by engaging in lots of discussion (in class and with visiting artists and interdisciplinary colleagues), I am hoping to grow their confidence to voice their ideas, as well as develop their own personal values regarding VR,” she said.
The nine students enrolled in the class were each provided with an HP Reverb G1 VR headset, provided to the Carson Center through their designation as an HP/Educause Campus of the Future.
“Being your average college student, I don’t have $500+ to spend on a new VR headset,” said Film and New Media senior Gabe Eubanks, who is taking the course. “Before class even started, all of the students enrolled were given an HP Reverb headset. It’s been my first extended experience with VR, and I have loved every minute of it. The class is a nice mix of concepts of VR and technical skills of implementation. Some classes we discuss the history, news and technologies around VR, while others we go about creating our own VR experiences through Unity.”
Henson said all Carson Center students have also received a laptop computer, as well, thanks to a generous gift from the Johnny Carson Foundation.
“This headset doesn’t just run on any computer,” Henson said. “You have to have a powerful computer for this.’
Eubanks wants to enter the video game industry, so he was interested in taking as many game-oriented classes as possible.
“The fact that we have multiple VR classes in my own school was super exciting to hear about,” Eubanks said. “This class appealed to me because I am a narrative storyteller but haven't had much of an opportunity for environmental storytelling. This Creating Augmented Worlds seemed like the perfect introduction to this.”
As part of the class, the students participated in the virtual New Frontier exhibition as part of the 2021 Sundance Film Festival, which showcases emerging media storytelling, multimedia installations, performances and films.
“Just negotiating the Sundance platform itself was an experiment in social 3D spaces,” Henson said. “They had to have an avatar there as well and navigate the online 3D world to figure out how to even access the films, which was challenging. But that brought up a big discussion in our class about how virtual experiences are distributed. It was really great for them to get right in the middle of that and see what the challenges were and what the opportunities are, and how the world is responding in real time to hosting large events during the pandemic.”
Emerging Media Arts junior Olivia Benson said it was an “interesting” experience.
“The projects were neat,” she said. “I ran into some tech issues with some of them, the glitching on experimental works was a little disorienting, but the concepts themselves were interesting. I think what I took from it the most is how new of a frontier VR is still. Even the coolest showcased works still have issues. Issues that I can implement to fix in my own works and strive to solve moving forward.”
Eubanks said he learned a lot from New Frontier.
“Sundance New Frontier had some really cool experiences,” he said. “I learned a lot of important lessons from the handful of experiences I attended—some good, some bad. Most importantly, I realized how big the VR market actually is, and how much it is growing. The sky is the limit for storytelling in VR, and we have only barely scratched the surface.”
The students also participated in a virtual workshop with new media artist and creative technologist LaJuné McMillian titled “Understanding, Transforming and Preserving Movement in Digital Spaces,” which explored issues of cultural representation, erasure and exploitation in these technologies.
In the workshop, students created 3D avatars and attached their own movements to these characters using the open source tools MakeHuman and Blender, as well as commercial tools such as DeepMotion Animate 3D and Unity.
Henson said the workshop gave context to this creative and technical process by discussing the ways motion capture and dance are used in popular media—in games like Fortnite and animated movies—as well as historically, specifically in relationship to race and cultural representation.
“We learned about the ways culture and identity can be exploited and erased through a critical engagement with the software tools and media, discussed what it means to ethnically represent and preserve movement, and put these practices into action by creating our own digital representations of ourselves,” Henson said.
McMillian creates art that integrates performance, virtual reality and physical computing to question our current forms of communication. She was previously the Director of Skating at Figure Skating in Harlem, where she integrated STEAM and figure skating to teach girls of color about movement and technology. McMillian currently teaches workshops across universities and public programs, building on her project, “The Black Movement Library,” which is a resource and archive of Black movement and dance, motion capture and 3D models.
“The LaJuné McMillian experience was really interesting,” said Emerging Media Arts junior Kayla LaPoure. “She brought in so many aspects of avatars and technological issues that I never even knew existed. Being a witness over a spokesperson for someone was an aspect of the workshop that I found incredibly moving. A lot of times I think I, and others, don’t really know how to uplift and support our friends who are different from us. LaJuné made it clear—witness each other.”
Joining Henson’s students in the workshop were advanced dance students from the Glenn Korff School of Music and acting students from the Johnny Carson School of Theatre and Film, as well as additional emerging media arts students in Assistant Professor of Emerging Media Arts Ash Smith’s class.
“My real hope is that we continue to do more interdisciplinary, collaborative work because virtual space impacts performers, as well,” Henson said. “If we’re doing motion capture, dancers understand embodiment and motion. So what can we learn as technologists and designers from other fields like dance, like movement, acting and storytelling?”
Two additional visiting artists participated in the class: Immersive and VR UX (user experience) designer Zach Deocadiz (https://deocadiz.com), who talked about user experience in social VR, and VRChat Worldbuilder Vaughn Schmidt (https://vowgan.artstation.com), who talked about the VRChat platform.
For their final projects, students are building their own environments and unique avatars for VRChat, and will gather together embodying these avatars in VRChat to experience each other’s worlds. After the course is over, these worlds will be able to be visited by anyone with a VRChat account, either on desktop or in a VR headset.
LaPoure said the class has been engaging.
“I have learned technical skills on creating augmented experiences and have been made aware of accessibility issues surrounding technologies in the industry,” she said. “It has made me think about how I can reach the widest amount of people with XR [AR and VR] experiences.”
Henson hopes her students see the vast potential to create meaningful experiences using this technology.
“I hope that they start to engage their own critical thinking around the incredible power that we have to create engaging, persuasive, powerful experiences,” she said. “I hope they also start to see the landscape of what is happening in the industry and bring that industry dialogue into direct contact with academic and artistic discussion. As they are being prepared to go out and work in this industry, now in college is an opportunity for them to ask every question they can think of and to really challenge themselves to question why. Why should we do this? How is this project inclusive or accessible? What are the ethical challenges? What problem are we trying to solve, and why is virtual reality an appropriate solution?”
Eubanks hopes to use his VR experiences in the Carson School when he pursues graduate study in game design.
“VR experiences are very unique as you are completely immersed into the experience both visually and acoustically,” he said. “Before I thought VR was kind of like a fun gimmick that wasn’t lasting. Now, I understand VR can be just as much of an entertainment staple as movies and video games. The future is bright. I know there is still a lot that I have to learn, but I know I’m not a total rookie with it all anymore.”