Scientists have developed a tool named PoseSonic capable of accurately monitoring upper body movements for individuals wearing smart glasses. A recent study suggests that future iterations of smart glasses could potentially replace optical cameras with sonar technology, utilizing sound for movement tracking. This sonar-based approach aims to enhance accuracy, privacy, and cost-effectiveness in the production of smart glasses.
Researchers at Cornell University introduced PoseSonic, a technology combining micro sonar, powered by CHIRP technology (a miniaturized version of the technology used for ocean mapping and submarine tracking), with artificial intelligence (AI) to create a precise echo profile image of the glasses wearer. The micro sonar captures sound waves beyond human hearing. The research was published on September 27 in the ACM Digital Library.
Cheng Zhang, study co-author and assistant professor at Cornell, emphasized the potential of this technology as a sensing solution for wearables, particularly in everyday settings. He highlighted the advantages over current camera-based sensing solutions, citing PoseSonic’s efficiency, affordability, unobtrusiveness, and privacy-conscious design. While existing augmented reality (AR) smart glasses rely on cameras for tracking wearers, this continuous video usage drains batteries quickly and poses privacy concerns. In contrast, acoustic-based tracking, as demonstrated by PoseSonic, offers a more economical and privacy-friendly alternative.
PoseSonic incorporates microphones, speakers, a microprocessor, Bluetooth module, battery, and sensors. The researchers developed a functional prototype for less than $40, a cost that could potentially decrease with mass production. In comparison, the Google Glass, introduced a decade ago, had a manufacturing cost of $152.
The speakers in PoseSonic emit inaudible sound waves, bouncing them off the body and back to the microphones. This data helps the microprocessor generate a profile image, which is then processed by an AI model estimating the 3D positions of nine body joints. Unlike other wearable systems, PoseSonic’s algorithm can work on any user without the need for specific training, as it is trained using video frames for reference.
Due to the lower power consumption of audio equipment compared to cameras, PoseSonic can operate continuously on smart glasses for over 20 hours. Future versions of the technology could integrate seamlessly into AR-enabled wearables without causing discomfort or excessive bulk.
Privacy is a key advantage of sonar over cameras, as the algorithm processes only the self-generated sound waves for building the 3D image, avoiding capturing external sounds or images. Additionally, data processing can occur locally on the wearer’s smartphone, reducing the risk of interception during transmission to public cloud servers. Cheng Zhang envisions practical applications for acoustic tracking in smart glasses, such as recognizing upper body movements in daily activities or tracking exercise routines. This technology could potentially offer users more detailed insights into their behavior and movement during physical activities beyond basic metrics like step count or calorie consumption.