New work shows that adding an inexpensive thermal camera to wearable devices could substantially improve how accurately they estimate calories burned.
The estimates of calories burned made by smartphones, smartwatches, and other wearable devices vary wildly. That’s because these devices lack the sensors required to gather all the information they need to make accurate estimates.
Using the thermal camera to monitor a person’s breathing rate and body temperature could reduce inaccuracies in energy expenditure estimates from nearly 40% with a current smartwatch to just under 6%, says Mayank Goel, an associate professor in the Carnegie Mellon University School of Computer Science’s Software and Societal Systems Department (S3D) and Human-Computer Interaction Institute (HCII).
“When people see these numbers, they make changes in their behavior and that can be troublesome if the numbers are wrong,” Goel says. Someone who thinks they just burned 400 calories on the treadmill, for instance, may eat more calories throughout the day, even though their actual expenditure was closer to 200 calories.
“That is a huge problem,” he adds.
Monitoring respiration has been a longtime interest of Goel and his Smart Sensing for Humans (SMASH) Lab, which develops technologies for such applications as health sensing and activity recognition. For instance, he previously developed ways to measure breathing using several methods ranging from wireless router data to custom wearables that analyze chest movements.
While pursuing a different project, Maite Sadeh, a Cornell University information sciences major who was a SMASH Lab summer intern, found reports on how respiration could be measured using a thermal camera to detect exhalations of hot air. Goel’s group then realized that inhalation leads to evaporation around the lips and nostrils. Both of these signals can be captured by a thermal camera.
Rishiraj Adhikary, a PhD student in computer science at IIT Gandhinagar who was also a lab intern via a Fulbright Scholarship, then found studies showing that respiration combined with heart rate could be used to measure energy expenditure.
But respiratory and heart rates aren’t sufficient because they fail to take individual physical and contextual differences into account, Goel says.
The gold standard for clinically measuring energy expenditure is a calorimeter, which uses heart rate, respiration, and the concentration of carbon dioxide in exhaled breaths to determine calories burned. Wearables already do a reasonable job at measuring heart rate and adding a thermal camera would provide a means for measuring respiratory rate. No solution exists for measuring CO2 concentrations with a wearable device, but the thermal camera could measure body temperature.
“We lose the ability to measure the concentration of oxygen and CO2, but we gained temperature measurements,” Goel says.
The combination of those three data points, with help from machine learning, enabled the researchers to develop a system, called JoulesEye, for estimating energy expenditure. They recruited 54 participants who either cycled or ran for 15 minutes. Their tests showed that JoulesEye could estimate burned calories with an error rate of just 5.8% when compared to a clinical calorimeter.
In addition to helping fitness buffs, JoulesEye could be used in sports training, as well as for monitoring people with chronic diabetes or cardiovascular disease.
The cost of incorporating a low-resolution thermal camera into wearable devices should be feasible, as these cameras are already available for $45 or less. But Goel says the team is working to incorporate an even lower-resolution thermal camera into the system, which could lower the price of the sensor. It would also reduce privacy concerns about a camera being routinely pointed at the user’s face.
The team also hopes to reduce the amount of time the thermal camera must be aimed at the user’s face. It now takes about 40 seconds to make the necessary measurements.
“Our goal is that the time it takes to check your watch should be enough time to get the information we need,” Goel says.
A report on JoulesEye, coauthored by Goel, Adhikary, Sadeh, and Nipun Batra, an assistant professor of computer science at the Indian Institute of Technology (IIT) Gandhinagar, appears in the Proceedings of the ACM Mobile, Wearable and Ubiquitous Technologies. The researchers will present their work in October at the UbiComp 2024 conference in Melbourne, Australia.
Source: Carnegie Mellon University
