Their ShadowSense system uses an off-the-shelf USB camera to capture the shadows produced by hand gestures on a robot’s skin. Algorithms then classify the movements to infer the user’s specific interaction. Study lead author Guy Hoffman said the method provides a natural way of interacting with robots without relying on large and costly sensor arrays: The researchers tried out the system on an inflatable robot with a camera underneath its skin. [Read: How Polestar is using blockchain to increase transparency] They trained and tested the classification algorithms with shadow images of six gestures: touching with a palm, punching, touching with two hands, hugging, pointing, and not touching. It successfully distinguished between the gestures with 87.5 − 96% accuracy, depending on the lighting. The researchers envision mobile guide robots using the tech to respond to different gestures, such as turning to face a human when it detects a poke, and moving away when it senses a tap on the back. It could also add interactive touch screens to inflatable robots and make home assistant droids more privacy-friendly. “If the robot can only see you in the form of your shadow, it can detect what you’re doing without taking high fidelity images of your appearance,” said Hoffman. “That gives you a physical filter and protection, and provides psychological comfort.” You can read the full study paper here.
