Expressive Robotics Is Breathing "life" Into Machines

As a pedestrian, you're used to interacting with traffic, but you often rely on human interaction – like hand gestures, eye contact and body language – to navigate it safely. But as driverless vehicles edge closer to reality on public roads, humans are faced with something that's still foreign to the general population: reading the intentions of robots and communicating their own intentions to machines. 

To better understand the communication between humans and automated robots, and ultimately build trust between pedestrians and driverless vehicles, Motional (a driverless vehicle company created by Hyundai Motor Group and Aptiv) is adopting principles from a budding field known as Expressive Robotics -- the study of how robots can respond to a scenario in the same way that we expect a person might.  

Paul Schmitt, Motional's Chief Engineer, and his team are researching the biological aspects of how humans interact with vehicles to make riders more comfortable with self-driving cars. By using VR, as well as taking cues from Disney's Principles of Animation, the team's goal is to make this human-robot interaction simple, familiar, and intuitive. 

I reached out to Schmitt to help explain this emerging field and how our very human tendencies can help autonomous vehicles operate safer and with less awkwardness for the pedestrians that interact with them.

Can you explain what expressive robotics is as a field and why it's important for human-machine interactions in an increasingly automated world?

Sure, imagine this. As you're about to cross the street, you turn to see an approaching vehicle. There is something different about this vehicle. Maybe it isn't driving like other vehicles. Maybe it isn't behaving like other vehicles. And as it approaches, there is something else. The inside seems hollow somehow. Wait. The car is empty. There isn't anyone behind the wheel. How would you behave? What would you do? How would you feel?

For pedestrians to feel comfortable interacting with driverless cars, the car's behavior ideally aligns with the expectations. In other words, the car would signal its actions and intentions in ways that people intuitively understand. 

To facilitate and optimize this human-car communication, Motional is developing and trialing principles from a budding field known as Expressive Robotics -- or more simply put, the study and practice of humanizing robots.

As a pedestrian, we are used to interacting with traffic, but many of us rely on human signals/interactions -- such as hand gestures, eye contact, body language, or the typical behavior of a human-driven car -- to navigate roads safely. Naturally, those human queues don't exist in a driverless car. A self-driving car would safely pull up to a cross walk, but the driver's seat would be empty. You wouldn't be able to make eye contact before crossing the street. 

Now take this scenario and multiply it by the number of pedestrians a single driverless vehicle may encounter in a day, a week or a year. Then multiply that again by the dozens, then hundreds, then thousands -- then millions -- of driverless vehicles that are expected to be on the roads in our lifetime. It's an immense number of interactions for which pedestrians don't yet have a clear set of internalized instructions, and that's ultimately because we're asking humans to do something that's still foreign: communicate with robots.

At Motional, we're working to solve this. Our goal is to make driverless vehicles a safe, reliable, and accessible reality, and central to our mission is ensuring consumers understand how our vehicles fit into their communities, and feel safe in their presence. Ultimately, we're working to make this human-robot interaction simple, familiar, and intuitive.

How is Motional drawing on the field to make human/car interactions more seamless.

Over the last year, Motional has researched real pedestrians in thousands of interactions with autonomous vehicles using VR. During testing, participants are immersed in a realistic world, an intersection with both human-driven and driverless cars on the road. The pedestrian is simply asked to cross the street when comfortable, then this is repeated for a variety of scenarios.  Each scenario is crafted from Expressive Robotics principles and consists of a vehicle expressing its intent to stop (or not) via expressive (human inspired) motions, lights, and/or sounds. Then we ask participants to rate how they felt about each encounter. 

Our initial results suggest that pedestrians respond positively when driverless vehicles exhibit expressive braking and early stopping -- and we are researching the effectiveness of these and other methods over repeated exposure. We plan to take the most promising signals -- the signals that allow a driverless car to most clearly communicate with a pedestrian -- and incorporate them into our future Motional vehicle designs.

We did find a few surprises from the research. We found that the number of conservative pedestrians outnumbered aggressive. Since the participants were drawn from an urban environment, we expected more aggressive. For some pedestrians, vehicle distance seems to be the key decision factor. For others, it is vehicle speed.  And for a smaller population, it is driver awareness interaction.  But perhaps most surprising is how many participants didn't notice the driver's absence.  While participants weren't told that the study involved AVs, cues for the AVs were clear (or so we felt): the vehicle outside had the AV sensor-laden look and on the inside the driver seat was empty.  

How did Disney's Principles of Animation become a resource and how has it been used in your efforts?

To help us achieve the goal of simple, familiar, and intuitive communication of AV intent, we found inspiration in, believe it or not, computer animation. Specifically, we found a well spring within the Disney Principles of Animation. These were first described in The Illusion of Life and are used by Pixar and countless other animation studios to breathe life into inanimate wireframe models. Richard Williams's book, The Animator's Survival Kit, was also a source of inspiration. We were surprised at how applicable many of the principles could directly be applied to robotics, even robots without a recognizable human form (e.g., torso, heads, arms, etc.).  

We utilized a few of these principles within the virtual reality environment in crafting several of the expressive autonomous vehicle behaviors.  Examples include expressive braking, expressive stance ("nose dip" and "tail raise"), expressive engine sounds, expressive braking sounds. 

How do you envision vehicles organically communicating with pedestrians and what will it take to get there?

Our vision is that autonomous vehicles will interact with pedestrians and all other humans in ways that are simple, familiar, and intuitive.  

We think a successful approach is to draw cues from everyday objects and integrate them into AVs. Rather than move like robots, the vehicles will move, not robotically, but in ways that are familiar and predictable.  Rather than ding, the vehicles' sounds will be directional and illustrative.   

Expressive robotics for driverless vehicles, then, must be an intersection of these two paradigms: designing technology that communicates its awareness, perception, and intended action specifically for human passengers and pedestrians.

As Laura Major, Motional's CTO, notes in her book, What to Expect When You're Expecting Robots, robots of the future will be social machines.  For vehicles to work in our communities, the vehicles must work with humans.

As we plan for a future where driverless vehicles are part of our everyday lives, this research is critical in helping the technology enter our lives comfortably, safely, and efficiently. 

This vision is bigger than one company. Indeed, it is bigger than our industry. More research is needed from our academic community in this area. This is why we are publishing our research results to share our findings, raise awareness, and spark more ideas. 

We've been able to move the driverless industry from the realm of science fiction to where we are today, and to continue on our path to a driverless reality, we must continue to include our pedestrians, passengers, and communities on that journey.