Cornell researchers use smartwatch speakers and mics for full-body tracking

Researchers at Cornell University's SciFi Lab have developed EchoMotion, a system that uses ultrasonic sound waves from a smartwatch's built-in speaker and microphone to track full-body movement. The system emits inaudible sound pulses and analyzes the echoes that bounce off the wearer's arms, legs, and torso. A lightweight AI model then reconstructs the positions of 12 major body joints.

The team tested EchoMotion on unmodified Samsung Galaxy Watch 6, Galaxy Watch 7, and Google Pixel Watch 3 models. With no prior calibration, the average joint position error was under 4 inches. After 12 minutes of personalized movement recording, the error dropped to about 2.4 inches. The system is under 30MB in size and draws roughly 12 milliwatts of power, using about 5 to 6 percent of battery over a 30-minute session.

In a gym demo on the Galaxy Watch 6, EchoMotion recognized 20 exercises such as dumbbell lateral raises and squats with 95.4 percent accuracy, even with background music at 76 dB. Accuracy did decrease when participants moved in a crowded hallway due to interference from other people, but furniture and road noise had little effect.

The researchers envision EchoMotion helping users track workout form, monitor daily activity, and assist with rehabilitation exercises. They are also exploring integration with smart glasses for body tracking in extended reality applications.