What did we learn from using Fitbits for a year?

Gathering personal data with wearables and activity trackers has promised to revolutionise healthcare, but what is the reality? Here at Reaktor, wearables have spurred both professional interest and first-hand experiences. In this first part of our wearable series, Juuso discusses our own experiments with Fitbit activity trackers.

A bit over a year ago we gave all willing Reaktorians a Fitbit Charge HR activity tracker. Our goal was to improve our employees’ well-being as well as learn more about the device and its possibilities. In this post I’d like to share what we’ve learned.

Picking up healthy habits

We did a survey amongst our people using the tracker. The responses messaged a clear positive effect: two out of three respondents felt that using the tracker has had a positive effect on their daily activity. The respondents mentioned changes in daily habits such as avoiding elevators and taking the stairs instead, and walking more in general. Some had also started paying more attention to exercising.

More than half felt that using the tracker had a positive effect on their sleeping habits. When they could see the data, they started paying attention to their sleep length and quality by going to bed early, avoiding coffee after 6pm, and reading a book instead of using their phones in bed. Many reported that their alcohol consumption has dropped due to visible negative effects on sleep quality and resting heart rate.

I think sleep was the most beneficial feature. I try to get 7,5 hours sleep a day and now I can track that.

Fitbits also encourage social activity in the form of various challenges, and in some or our teams these became really popular, boosting the positive effects of the devices. There were also indirect benefits, as we started to discuss health-related stuff in general much more actively than before. People started sharing good tips and their personal experiences with others.

Not without issues

Naturally there were also less positive experiences. Some people refused to even start using the tracker, and many have stopped using it over time. Most of the negative comments were about the tracker hardware. Some considered the tracker too bulky or ugly, whereas others had problems with skin irritation or poor battery. A notable share of the devices have broken during the year, and Fitbit Charge HR indeed has an extremely high return rate at our local retailer.

I used it for a few weeks, but then stopped due to a combination of not getting enough value, not being able to use watch, and the device feeling too thick.

On the software side, issues with syncing were reported. The syncing often took insanely long, got randomly interrupted, or simply didn’t start at all. The user interface also spurred a lot of reactions, especially among our own UI specialists. However, both syncing issues and UI design were improved notably over time by Fitbit.

There were also problems with inaccurate sleep length tracking and absurd floor counts. Sleep length is not always indicative of sleep quality, and Fitbit does not distinguish between light and deep sleep, as many other devices nowadays do. Compared to traditional heart rate monitors, the heart rate values seemed to be reasonably accurate during rest, but quickly became very unreliable with elevated heart rates during exercise.

Surprising algorithms

In addition to the basic functions provided by Fitbit tracker and app, we explored whether we could collect and analyse the data from the devices by ourselves. Getting the daily statistics, such as resting heart rate, sleep time and step count from the Fitbit API is straightforward. It’s also possible to get more detailed intraday data, but this requires separate permission from Fitbit.

Perhaps the most interesting single number provided by Fitbit is the daily resting heart rate (RHR), which is commonly used to describe the health of your heart. This is what most Fitbit users follow actively, and we also had plans to study associations between resting heart rate and other factors, such as amount of sleep or exercise. Maybe we could even predict when someone was in danger of a burnout or catching a flu.

However, the Fitbit RHR is computed with a proprietary black-box algorithm, and we quickly realised that it’s not the commonly used and simple average heart rate after waking up. Instead, the RHR value is sometimes updated during the day, or even afterwards. Values in successive dates also seem to depend on each other, suggesting some kind of autoregressive smoother.

Around last March many of us observed a notable decrease in our heart rate values. This was also visible in our collective data (see figure below), providing strong evidence that Fitbit had changed their RHR algorithm.

The figure shows a clear drop in the daily resting heart rate values (bottom) around March 2016, whereas daily minimum values (top) are not affected. This indicates that something changed specifically in the way the RHR is computed. Btw, can you spot our annual company-trip? Hint: look for high peaks in heart rate values.

 

This was a disturbing finding. The change in the RHR algorithm resulted in lower values for many users, who might falsely interpret that their health had improved. Moreover, such drastic changes unfortunately undermine any attempts to use the data for longer term statistical analyses.

Clear use cases needed

Overall, we have quite mixed feelings from the experiment. In terms of data, we were disappointed, especially in the problems with the resting heart rate algorithm. Also the initial syncing problems of the devices and their tendency to break down led many of us to abandon our Fitbits soon.

On the other hand, for many the device was a positive experience. We nudged ourselves towards healthier everyday activities, and gained a lot of valuable experience. However, it remains unclear how long the benefits will last, as many studies have shown how the benefits dilute over time, and this is also visible in among our Fitbit users.

It’s also very important to realise that the individual experiences and benefits vary a lot. This is naturally affected by the goals and expectations, life situation, and overall condition of each individual. So far we have only observed very general effects, which are ambiguous at best.

The trackers may well benefit specific use cases with measurable outcomes. However, identifying and validating those cases needs both a better understanding of the users’ needs and careful experimentation. There’s also a lot of room for more intelligent applications that make conclusions and suggestions for the user based on the data.

The discussion on wearables will continue with a Part II, as Krista will bring a designer’s perspective on the table. In her blog post, she talks about how wearables could be more user-friendly and thus even more beneficial in the long run.

Oh, and have you already seen how we reinvented the Sports Tracker app? Or helped Suunto succeed in their biggest software project yet?

Recommended posts