Personal Analytics: The Next Big Thing in Self Improvement?

Recently the idea of personal analytics hits me more and more. What is personal analytics? It’s a way of making decisions in someone’s life based on numbers. You measure certain aspects of your life, analyze the results, and make decisions based on them.

Over time I find more things pointing to this direction. All these make me wonder: is personal analytics the next big thing in self improvement?

It started about two years ago when I read an excerpt of the book Super Crunchers. The book talks about how smart companies “act by numbers.” Instead of making decisions based on intuition, they make decisions based on what they measure and calculate. For instance, they use mathematical models to accurately predict how the customers will behave in certain situations and adjust their operations accordingly. This way the companies can directly make the right decisions without wasting their resources. The book even says that those companies know about your preferences better than you.

Reading that made me think: is it possible to apply that at individual level? Can individuals live by numbers? Can someone make decisions for his life not based on intuition but based on facts? If we can apply that, that will be a great leap forward. Just like the companies, you will be able to optimize your life to get the most out of it.

As it turns out, some people already do that. They already live by numbers to optimize their lives. My favorite example is Jim Collins, the author of Built to Last and Good to Great. Here is what The New York Times wrote about him:

And in a corner of the white board at the end of his long conference room, Mr. Collins keeps this short list:

Creative 53%

Teaching 28%

Other 19%

That, he explains, is a running tally of how he’s spending his time, and whether he’s sticking to a big goal he set for himself years ago: to spend 50 percent of his workdays on creative pursuits like research and writing books, 30 percent on teaching-related activities, and 20 percent on all the other things he has to do.

These aren’t ballpark guesstimates. Mr. Collins, who is 51, keeps a stopwatch with three separate timers in his pocket at all times, stopping and starting them as he switches activities. Then he regularly logs the times into a spreadsheet.

That’s not all. From the same article:

Oh, he sleeps with vigor, too. He figures that he needs to get 70 to 75 hours of sleep every 10 days, and once went to a sleep lab to learn more about his own patterns. Now – surprise, surprise – he logs his time spent on a pillow, naps included, and monitors a rolling average.

“If I start falling below that,” he says, pointing to the short list on his whiteboard, “I can still teach and do ‘other,’ but I can’t create.”

Isn’t that amazing? He carefully measures certain parts of his life in order to improve them. He makes his decisions not based on memory or intuition but based on numbers. Jim Collins is the best example I’ve found so far of someone who is living by numbers.

But will personal analytics be a common phenomenon?

Well, there is one difficulty here: the data-gathering process. A company has a lot of resources to measure almost anything in its operation. It can set up a separate division to do just that. But an individual has limited resources. You have only yourself to do everything. I’m sure many people won’t be patient and diligent enough for that. Even those who do must be careful not to spend too many resources on it at the expense of the real, productive work. Because of these difficulties, only a few people like Jim Collins apply personal analytics effectively.

Fortunately, it may change in the future. There are new products coming out that make it easy for you to gather data. One good example is the Nike+ system:

Veronica Noone attached a small sensor to her running shoes and headed out the door. She pressed start on her iPod and began keeping track of every step she took. It wasn’t a long run – just 1.67 miles in 18 minutes and 36 seconds, but it was the start of something very big for her.

Since that day, she’s run 95 more times, logging 283.8 miles in about 48 hours on the road. She’s burned 28,672 calories. And her weight, which topped 225 pounds when she was pregnant, has settled in at about 145.

Noone knows all of that thanks to the sensor system, called Nike+. After each run, she can sync her iPod to the Nike+ Web site and get a visual representation of the workout – a single green line. Its length shows how far she’s gone, and the peaks and valleys reflect her speed.

Products like Nike+ make data gathering effortless. We can expect more and more products like that emerge in the future. When that happens, you can easily measure many aspects of your life and use them to optimize your life.

Photo by Hexadecimal Time

16 Comments

  1. Beautiful insights. I think micro-feedback loops are the key to change.

  2. I totally agree with you that measuring helps you make better decisions. Although I find the act of keeping record of how I spend my time is a little onerous. Will be good when some technology comes along that makes this easier.

  3. Interesting post. I’ve subscribed to web-based tools like DAYTUM and YOURFLOWOINGDATA but can’t seem to motivate myself to collect what i might consider to be important.

    I think a careful plan to measure only a small subset of important information is perhaps the trick.

    I also like the idea of using a very flexible visualization tool (like Tableau Software) to interpret what is going on versus these web-based solutions, which are visually appealing but involve a little effort to learn to use effectively.

    I like the stopwatch idea. Having (3) wouldn’t be that costly. I think the key is to make it easy to update and limit the things you measure.

    Nice post. Thought provoking.

  4. Consider the ramifications of the “by the numbers” system before you leap in with both feet. It’s fine for some things, but has the most terrible consequences for others — analyze your relationships and friends by the numbers? Evaluate your co-workers and fellow citizens by the numbers? Find the formula for perfection and work up a quantifiable step at a time? The concept came out of the Enlightenment, when science began to explain the world “by the numbers”, resulting in formulas for human perfection taken to the extreme by fascism, communism, national socialism, Pol Pot, to name a few. Our society is awefully close to this now – “a few eggs must be broken for the good of all – by the numbers”.
    Don’t go there!

  5. I like the tie in with the book and the use of a journal. I like to keep track of the past to see how far I have come. By the way the book super crunchers was great. Have a great week

  6. This makes a lot of sense but must be tough to follow. Especially gathering stats and figuring out what works part.

    At the same time, feedback that numbers provide is so powerful. Numbers gives us room to plan and gauge things.

  7. J.D.,
    Micro-feedback loops… love that term 🙂

    Kaizan,
    That’s the problem we now have. I myself hope that it will be easier over time.

    Dan,

    I think a careful plan to measure only a small subset of important information is perhaps the trick.

    I agree with you. We should collect only the data that can help us make the most difference. By the way, looks like I need to check the tool you mentioned.

    Randy,
    There is indeed danger with this approach. That being said, I still believe it’s good for some aspects in life. Time management and energy management come to mind.

    Paul,
    Yes, the book is an eye-opener. Have a great week too.

    Avani,
    What I love about numbers is they can give us awareness. Sometimes we don’t see how serious a problem is until we see the numbers.

  8. This is one heck of a great post. All the very good business people I respect do this both for themselves and their business. JB Glossinger @morningcoach, does this for himself, recording his spending and eating habits daily. @websuccessdiva is meticulous about using Google Analytics, and @timferris is the same for his site and business. Being more right brained, I struggle with this, but need to marry the right brain with the left side of my brain 🙂

  9. An advancement indeed is what i call it. Living my numbers and not by intuition is what i believe to be a next level, when it pertains to strucring ones life. Yet, it may be confusing for some at first it can create an organized life for millions.

  10. Steve,
    Nice examples. Personal analytics is still relatively new, so most people will need time for transition.

    Jonathan,
    I agree. If it works for individuals as it works for companies then it can greatly improve many people’s lives.

  11. Great post! I can certainly attest to the fact that measuring progress objectively is great for making more progress. I’ve done several experiments with measuring different aspects of my life already and it was often very interesting. The biggest problem is keeping up with all the tracking. For example, I also used stop-watches to time different activities for a while, much like it’s described in this article. I couldn’t keep the habit up, though, because it’s quite a bit of bother to always remember when to stop one and start the next timer and then logging all of that. Automatic tracking devices are certainly the way to go here.

  12. I don’t think I can agree with any of you guys. Everything in life “should” not be perfect. I consider my life (driven more by intuition; numbers only get to help) very close to perfect compared to others. Now if everyone gets perfect, then there is no scale of comparison, and that becomes level-zero then, then none of us are perfect, back to square-one!

    Also, maybe all the math models will fail if they are all pitched against each other!

    And besides, our heads are all neural networks, which can basically do anything, and only 2% of their total capacity is utilized. We just need to put these network to more work, and I am sure that in the case of man, Darwin’s theory will kick in and act magically quickly in a couple hundred years. There will be no need for computers then.

  13. […] (follow me on Twitter) , March 15, 2010 Advertisements A few months ago I wrote a post on personal analytics. There I discussed how more and more people make their life decisions based on numbers. They […]

  14. I have made a small Pomodoro app called CherryTomato – http://www.beatpoints.com/cherrytomato
    When you finish a pomodoro (see http://www.pomodorotechnique.com if you are not familiar with it), you can rate it. This information is stored, so at some point I am hoping that this data will be interesting to mine – when am I being productive? Comparing this to say, sleep data or eating habits could perhaps give me some insight into what really affects my productivity..

  15. It’s brilliant to come across like minded people!
    We’re just developing tool for doing that in unobstructive and simple as you can only imagine way! Please sing up for beta testing on http://www.StatsCollector.com and help us to build it!

  16. interesting post!

    I wonder how many of us would actually take self improvement to the extend like Jim Colins did?

    when i look around i find most people aren’t even honest in filling out a timesheet at work

Comments are closed.