Researchers at Cornell University are developing a camera solution to enable soft robots to sense human interaction through shadows.
The study has resulted in a low-cost method for soft, deformable robots to detect a range of physical interactions, including pats, punches and hugs, without relying on touch.
Alternatively, a USB camera located inside the robot captures the shadow movements of hand gestures on the robot’s skin and classifies them with machine-learning software.
The ShadowSense technology originated as part of an effort to develop inflatable robots that could guide people to safety during emergency evacuations.
The team said such a robot would need to be able to communicate with humans in extreme conditions and environments, and looked to sight in order to gauge touch.
“By placing a camera inside the robot, we can infer how the person is touching it and what the person’s intent is just by looking at the shadow images” said doctoral student Yuhan Hu, lead author of the study.
“We think there is interesting potential there, because there are lots of social robots that are not able to detect touch gestures.”
Hu’s team created a prototype robot that consists of a soft inflatable bladder of nylon skin stretched around a cylindrical skeleton.
It is roughly 4ft in height and is mounted on a mobile base. Under the robot’s skin is a USB camera, which connects to a laptop.
Furthermore, researchers developed a neural-network-based algorithm that uses previously recorded training data to distinguish between six touch gestures.
This includes touching with a palm, punching, touching with two hands, hugging, pointing and not touching at all. The trials are said to have had an accuracy of 87.5% to 96% depending on the lighting.
Hu added that, by collecting enough data, a robot could be trained to recognise an even wider vocabulary of interactions, custom-tailored to fit the robot’s task.
The technology can also be applied to other equipment and be incorporated into other materials, such as balloons, turning them into touch-sensitive devices.
Additionally, as the robot only senses a shadow it offers privacy, according to Hu. “It can detect what you’re doing without taking high fidelity images of your appearance.
“That gives you a physical filter and protection, and provides psychological comfort.”
The study was published in Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies. Â
