MIT engineers build soft shape-sensing skin for gaming and VR

Soft sheet tracks its own 3D shape

MIT engineers have built a soft wearable sheet that tracks its own 3D shape and touch, a design MIT News says could be useful for controlling games, virtual reality and physical therapy. The work follows a paper in Advanced Intelligent Systems titled Toward fully soft and multifunctional shape sensing via optical waveguide arrays. The sensing surface is fully soft, although the sheet does not stretch as a whole. An external circuit board collects data and an algorithm turns it into a full sheet and 3D shape. The device uses 14 waveguides, each 160 mm long, in two layers of seven zig-zagged inside a 16 x 16 x 0.5 cm silicone sheet. An LED sits at one end of each fiber and a light sensor at the other, with electronics at the edge, including analog-to-digital converters feeding microcontrollers over I2C. Reconstruction runs in MATLAB about three times a second, including sensor sampling, data logging and plotting.

Fibers read bending direction

A plain fiber has a transparent core inside light-trapping cladding, and less light reaches the far end when the fiber bends, so the light output helps measure the bend. MIT replaced glass with a flexible rubber core and used black-dyed cladding. One side is roughened on purpose, so bending toward that side or the opposite side causes different light loss, enabling bi-directional sensing. The fibers also respond to pressure and can sense external force such as a press on the forearm in a physical therapy demo. The force sensing has no calibrated force reading and runs separately from the shape mode. The team simulated sheets with 3 to 20 waveguides per layer, at angles from 0 to 35 degrees, on six target shapes with 10 percent added noise over 30 trials each. It found that angling the waveguides improved accuracy more than adding more of them, and selected the zig-zag spacing with the closest fit to known shapes.

Accuracy, resilience and uses

In lab tests the sheet kept its error below half a centimeter. For accuracy, it was laid over three 3D-printed molds of known shapes, and the root-mean-square distance between real and reconstructed surfaces was 0.37, 0.28 and 0.45 cm. MIT mechanical engineering graduate student Qifan Yu put rigid-sensor designs at about 1 to 2 cm and called the result at least on par with existing technologies. Damage tests showed the rest could still reconstruct the overall shape after fibers were cut or disconnected. In simulation, error stayed under 0.3 cm with up to three of the 14 waveguides disabled, then rose to about 1 cm or more from five on. Wearable demos were done by one author on their own body, with simpler scaling for wrist and elbow angles because stretching at those joints throws off the shape algorithm, and comfort was not commented on. Yu said the setup targets fully soft sensing surfaces, so it is potentially safer around people. MIT assistant professor of mechanical engineering Kaitlyn Becker, a co-author, said the team hopes to help physical therapists track and quantify patient progress over time and compare sessions more precisely. The technology could also mimic motion in virtual reality, support teleoperation of robots or be used in soft robots. The researchers plan to target a thinner sheet first. Today the fibers are 1 mm thick, and they could reach tens of micrometers, thinner than a human hair, with a higher fiber count enabling smart garments. Future work includes a faster solver instead of the general-purpose MATLAB one, more waveguide layers and narrower waveguides for better detail with thinner sheets, a separate sheet to gauge force and shape simultaneously, and stretchable electronics.

Sources: Tom's Hardware