Advanced aGent Engagement Team (AGENT) explores the world of virtual reality

Last Updated on June 16, 2017 at 12:00 AM

Originally published June 16, 2017

By Lauren Macias-Cervantes

UTEP Communications

Whether you like talking to machines or not, artificial agents will be part of our future. The Advanced aGent ENgagement Team (AGENT) at UTEP is working to make sure your interaction with these virtual personalities is a worthwhile experience.

Ivan Gris, Ph.D. helps Michelle Cromer with the headset that accompanies one of the team’s systems. Cromer was one of several people that attended an open house of the lab in April.

“Artificial agents are an increasing part of our lives, whether you are shopping online or renting a car, getting service and I, for one, want to see that agents that are more useful to me, more natural to me and avoid having to make me adapt the way I talk,” explained David Novick, Ph.D., professor of engineering leadership. “I want them to be helpful and to let me talk the way I want to talk.”

Novick formed the team in 2012, and since then they have published 20 papers on interaction in immersive and virtual environments. They have created tools for building agent-based systems and used these tools to develop several applications.

“The long term goal is to create artificial agents who are helpful and natural to talk with or interact with humans,” said Novick. “Right now when people work with machines we have to adapt ourselves to what the machines are doing. We are trying to create artificial computer generated systems that are more natural for humans to deal with. We want to move the interaction over to the human side instead of having people act like robots. To do that we run experiments about what is natural or what creates rapport, what creates engagement, and in order to run those experiments we have to build systems where we can actually use them.”

UTEP College of Business student Monica Pena has experienced many of the team’s creations.

“It is incredible to see how technology is advancing so fast and even more fascinating to see it right here at UTEP,” said Pena. “It is very inspiring to have a group of people at my university create such amazing things.” 

Pena has interacted with a variety of multiple agents, but has a couple of favorites.

“Notably, I experienced a demo of a haunted house, which was created for Halloween a couple years ago. It had jump scares and music that would most definitely set the haunted ambiance. The rooms were creepy and made me unsure as if I actually wanted to finish the game. It felt real, as you controlled your gaze and steps,” Pena explained. “The Merlin demo was definitely a good one as I was able to help him in his quest of destroying a monster by actually calling out spells with my own voice. I walked through a forest along Merlin, who really did look like the character. Although I used it some time ago, I still remember the stress and reality of seeing a huge monster standing in front of me as I anxiously screamed the spells, hoping to defeat him, which I successfully did. It definitely does bring a sense of accomplishment. My friends still make fun of me because I kept screaming ‘Inflammo!’ so loud.”

In 2015 the team received the award for Outstanding Demonstration at the 17th ACM International Conference on Multimodal Interaction a system for virtual agents and an immersive interactive application titled “Survival on Jungle Island.”

In “Survival on Jungle Island,” an embodied conversational agent (ECA) and a human interact, using speech and gesture, in a 40-60 minute adventure composed of 23 scenes. A study conducted with the adventure showed that rapport increases when the ECA asks the human to perform task-related gestures and then perceives a human performing these gestures.

In the jungle adventure, the system simulates a survival scenario in which the player interacts with the ECA, Adriana. To survive, both human and ECA must collaborate, cooperate, and build a relationship.

“We’re trying to help educate students,” said Novick. “It’s exciting and fun, but there’s a serious educational component. There’s a serious research component. We are really doing fundamental research about the nature of interaction.”

Doctoral candidate Adriana Camacho has been a student on Novick’s team for four years and helped to start an organization for students called GameBuilders!. Camacho said students work collaboratively to improve agent connections. 

Recently the team’s lab held an open house to showcase four systems developed collaboratively on campus.

“Gods in the Neon City”

  • The virtual reality gaming system will be used this fall to determine how to improve rapport between people and artificial agents.

“Boston Massacre History Experience Project”

  • The educational experience for 8th graders was developed jointly with Brad Cartwright, Ph.D. in the Department of History with graduate student Laura Rodriguez.

“Recipe Hunt”

  • The interactive experience, developed for the Smithsonian Latino Center, involves the search for the authentic recipe for chile con queso, developed in cooperation with Sylvia Khan, Ph.D., in the Department of Communication.

“Movies/Vacations”

  • The immersive application for study of agent naturalness allows participants to chat about favorite movies and vacations with an artificial conversational agent.

Visitors had the opportunity to experience the simulations and interact with a life-size virtual agent.

“We are building these agents to see if we can connect with them,” Camacho explained. “With educational agents, we can determine if students connect with the agent and learn much more than just using a textbook. In the recipe system, people can connect with the agent and learn more about the culture and more specifically about recipes in El Paso.”

Human-computer interaction is the graduate student’s focus. She specifically helped the agent appear more lifelike by developing her breathing in a few of the scenarios.

“If they’re more natural then maybe we can relate to them much more.”

More information about the team’s work is at isg.cs.utep.edu.

“People are excited about virtual reality and that’s great to keep the momentum for something more impactful,” said Camacho. “Entertainment is great and we all want to have fun, but if you can have fun learning, that’s even better.”