Computer scientists just developed a system for helping AI understand human goals

Their paper describes a system that uses two-way communication in (somewhat) natural language to help AI-enabled robots understand what their human operators actually want them to do. The researchers used a game to test and fine-tune their system, but their ultimate goal is to create a general AI capable of helping humans with all kinds of tasks. Interesting Engineering sat down with computer scientist Luyao Yuan, one of the researchers, to talk about what his team is doing, how their current system enables human-machine collaboration, and how such a system might be helpful in the future.

This interview has been edited for length and clarity.

Interesting Engineering: At the most fundamental level, what problem are you trying to solve?

Luyao Yuan: We aim toward generic human-robot collaboration. We always make the comparison to the kind of human-human collaboration that happens every day in our daily lives. For example, teammates have aligned goals and intentions so that we can have effective collaboration. 

If we want to have a system that can help us achieve successful performance of tasks, we need to make sure that the robot has the same intentions and understands our objectives. Currently, those things are coded into most AI systems. These are either coded in by experts or by a whole group of engineers that design what the AI system should do. Those things are pretty much fixed. We cannot easily adjust a robot. It's not that flexible.

IE: How have you gone about making a system that's flexible enough for people who aren't experts in AI to use?

We're enabling the robots to learn the user's objectives and intentions from the user's feedback in real-time during the collaboration. That way, we don't have to code in anything beforehand. We're developing something like a normal assistant, like a human friend. You don't need to spend hours aligning before starting to work together. Most of the time, humans align during collaboration. We enable that capability in robots so that they can understand the users and make sure the user understands them. So, it's a mutual understanding between humans and robots during collaboration. That's why we call it a dynamic explainable artificial intelligence system. We enable the whole thing by learning from a human's feedback.

IE: What's your goal for this system?

We have one robot that should be able to help the commander achieve all types of goals. The robot doesn't know the commander's goals because they're just so diverse. We want the robot to be able to infer the commander's goals during the game.

IE: The game you present in the paper seems very complex. A map overlaid on a 400-square board includes obstacles, rewards, and hazards. What's the objective of the game?

The human commander has a certain set of goals, such as exploring the whole map as quickly as possible, saving time, collecting resources, and diffusing hazardous substances. Every goal is valuable, but there could be trade-offs. Sometimes you don't care about time, you just want to make sure the whole map is clear. Sometimes you don't want to explore the whole map, you just want to pass a certain area as fast as possible. The robot doesn't know the commander's objectives. It needs to infer the commander's values through interaction.

IE: How does the AI help the commander play?

The map is complicated. Although the commander knows the ultimate goals, the commander doesn't have the mental capability to infer the optimal trajectory to finish the task. That's the responsibility of the robot. After combining information from the map and feedback from the commander, the robots do a bunch of computations to come up with optimal trajectories. Then they propose the tentative path plans to the commander, who will simply say "yes" or "no" to those proposals. 

It forms a loop. The robot has a certain estimation of the commander's value. Then it makes plans based on that tentative estimation of the value and it gives the plan to the commander. The commander gives feedback to the robot, and then the robot will update its estimation for value based on feedback. Then it makes new proposals. It keeps iterating until both of their values align and the robot and commander understand and trust each other.

IE: It seems like you're developing a system that plays to the strengths of the humans and the robots. Is that right?

Yes, but we don't emphasize the cognitive contribution from the human. We're actually trying to reduce the cognitive burden for the commander because the human mind only needs to make relatively simple decisions. When the environment is complicated, the robots need to take over and do the hard computational work. Usually, a human cannot come up with the optimal plan.

IE: What part does the human play in this game?

The human's job is to make sure the ground truth value is exactly the value the robot is using when the robot is planning. That is the human's role. So the human needs to guarantee, "Okay, the robot is doing what I'm expecting them to do," instead of some random thing that just doesn't make any sense.

I won't say it's like a combination of human intelligence, but we'll say it's a generic way for humans and robots to collaborate. We want the robot to help us in the way that we want, not in some random way that we don't want. That's the human's role.

IE: What are some practical applications for this explainable AI technology?

One imminent application that we can think of is self-driving cars. Every day, when we get up, we need to go to work. But your needs every day might be different. For example, right now, the gas price is so high. So today, I just want to choose [a route] that is the cheapest. I don't want any detour; I just wanted the cheap road. Maybe one day, I get up late, and I want to get into work as quickly as possible without worrying about the cost. Or maybe I just want to go somewhere with a good view, and I don't care about time, and I can spend some extra money. I just want to have a good view when I drive.

Or sometimes I want a balance, for example, 30 percent view, 30 percent gas efficiency, and 40 percent time. The balance can be very tricky to define, and sometimes the user might not be able to verbally specify the need for that day.

Sometimes I drive for leisure, sometimes I drive to work. If we have a self-driving system that can you infer the use or the passengers' needs on the fly, we can save time or prioritize cost. Maybe after three questions, the system knows what want, and you don't need to verbally say. Then it can optimize the trajectory, the roads, and then choose the best route for you to travel

IE: What challenges did you face in developing this system?

One challenge we had is that when people play the game, they don't like to be interrupted. They want to have a continuous experience from beginning to end. But for the experiments, we need to probe the participants during the game. Otherwise, there's no way we can understand how well the human and robots are aligning with each other, right? So we needed to come up with a way to probe the participants without interrupting the game too much. 

We had a computational-level challenge, too. We try to make sure the whole alignment happened in real-time rather than collecting a bunch of data too late. We wanted to make sure that the real-time alignment was happening. To overcome that challenge, we integrated cognitive science findings — an idea called theory of mind — into our computational model. Then we observed that alignment that previously could only happen with large data training is enabled with real-time interaction.