Researchers Creating Games for Orangutans with Xbox Kinect

On Monday, researchers began a project to understand how animals relate to interactive technology by making video games for orangutans with the Xbox One’s Kinect.

The teams from the University of Melbourne’s Microsoft Research Centre for Social Natural User Interfaces and Zoos Victoria created a game for the Xbox Kinect called Zap. They previously allowed their test subject, Malu, to play a simple tablet game, but it was clear that he wanted to do more than just tap a screen.

Zap uses the Kinect’s 3D sensor to project a game directly onto the floor of Malu’s cage. Interaction is as simple as touching the projected colors. Malu likes to kiss the projection, which causes a splash of light in reaction. The newest version rewards the orangutan when he and the human with which he is working touch corresponding spots.

The researchers want future games to attune themselves to the orangutans’ interests. When Malu tried to use a cloth to wipe at some of the colorful spots, it inspired them to make a game in which he could “sweep” ripples through the projected images.

These experiments are far more than simple gimmicks. Not only do they allow scientists to gauge the mental capacity of their subjects, they provide keen insight into the psychological states of the animals being tested. And perhaps most importantly of all, they provide an engaging and entertaining manner of recreation for creatures not constantly preoccupied with their basic survival, as they would be in the wild.

“We don’t just want to stop here,” Animal Welfare Specialist Sally Sherwen said. “If the trial is successful we could utilize digital enhancement to benefit a range of species, but this will be different according to a species’ preferences.”

Happier, healthier, and more engaged animals open channels through which we might be able to resolve basic communication. Animals, like infants, have little to no ability to express their needs or concerns. Interfaces that begin as games might someday help us understand the thought processes of the creatures all around us, crossing a Dolittle-esque communication barrier.

If nothing else, it looks like Microsoft has finally found an audience for the Xbox Kinect.

Follow Nate Church @Get2Church on Twitter for the latest news in gaming and technology, and snarky opinions on both.