It has been 9 months since I first shared experience to track tennis performance with Apple Watch. Backing up by popular demand(Surprised so many visitors found this blog from search engine all over the world), I’d like to take it further with a more in-depth review, of my own experience tracking and analyzing my tennis workout with the Swing app. Read More »
This blog post is aiming to provide a step-by-step guide to perform advanced analytics on swimming data, captured by Apple watch. Microsoft PowerBI and Python on Jupyter Notebook are the primary tools to prepare, analyze and visualize the data.
You will learn how to export the workout data efficiently to your PC, make necessary data transformation, and understand what metrics and dimensions are available. Then I will walk you thru how to analyze the data to answer typical questions related to why certain behaviors happened. You will then see my preliminary attempt to use advanced analytics tools to predict future swimming performance.
Most importantly, you will find quite a few reference articles related to this topic, hopefully fulfilling your intellectual curiosity.
It is also the #3 articles of a series, the previous articles can be found here:
The “life-long learner” is a pretty sexy title.
Over the past few years, I have received feedbacks, from both trusted and the anonymous sources, that self-improving seems to be one of my characters.
As a normal human being, I was flatted; As an analyst, I can’t control the skepticism: Is this a filtered bubble?
What does it really take, to become “life-long” learner? – Let me tell you my version:
Learning doesn’t equal to passing exams
Recently I have just gone thru a piece of an unpleasant learning experience. It was a government-sponsored exam aiming for testing my knowledge in a certain domain. I was provided with a handbook with over 60 pages.
The final outcome is great – I aced the exam. However, the overall experience was awful at best. Stressed, fear(of failing the exam), frustration(over less than ideal simulation test result), self-doubt(my my own identity), all of these negative feelings has crowded me, and I was forced to temporarily give up those roles that I am proud of, a husband, a parent and a dedicated employee, during that 2-day exam prep period.
Upon reflection, it has triggered the negative learning experience I had in my life growing up – memorizing a lot of things and aim for a high score in the exam in the exam.
Learning things like this is dreadful. Getting a good score on the final exam is important. However it should be the outcome, not the guideline dictating all the steps along the way.
What is the right learning experience？
I decided to take the time to think, reflect and organize my thoughts in order to write this post – a bit longer than usual. It can be breaking down into the following seven section:
- Experience is what matters the most
- The drive: balancing your 2X2 quadrant
- Choose your own “T shape” learning channels
- Money should not be your constraint
- Practice! Use it or lose it
- Reflection: continue optimizing
- Community: avoiding the one-man show
Experience matter the most
Experience is our best teacher. We are learning either thru our own experience (ex. try and error) and learning thru others’ experience(eg, books or other media). The latter one is usually much more difficult.
A recent example- I was recently invited to a “board game Friday” – a group of colleagues just play a board game together during lunchtime every Friday. The organizer always brings a new game, and he usually starts by explaining the rules in length, based on his own experience. I never felt truly understand it and usually quickly lost my attention. Once we are starting to play, it won’t take me long to figure out the rule and how to navigate it, by simply making a few errors, or observe how others play. In short, in this board game example, it is much easier to learn thru own experience, than from other’s experience.
On the flip side – “easy” doesn’t always translate to “right”. The evolution theory is built upon “learning from others” – most of the animals can learn from own experience as well, but only human can learn from others. Learning from others, in the form of observing, reading, writing, teaching, reflecting, help and contribute to the human evolution. At the end of the day, Team works always win over individual excellence.
It needs to have a balance between “learning thru try and error” and “learning thru others”. There is established framework claiming that 70% learning on the task(learn thru self), and 30% thru others (20% from peers and 10% from formal training). This optimal mix should be tailor to yourself.
The Drive: Balancing the 2X2 quadrant
Everything happens for a reason. Why are you choose to learn this? It is always a good idea to start by being honest with yourself, and understand what truly drive the behavior. In many situations, it can be implicit and happens unconsciously.
Research shows there are four main type of motivations
The top left corner – “positive” + “internal” driven is claimed to be the most sustainable drive for your learning experience. Curiosity is a simple example that fit into this category.
Not everything in the bottom right corner is “bad”. For example. Deadline is another one that leans toward “negative” and “external driven”. It is THE most effective way of driving actions not only myself but also quite a few of my friends. Despite being stressful, we often learned the most during the short period leading toward the deadline.
Knowing this, we can even set up our own time constraints, for example, making a public commitment to others. It just needs to be handled properly to avoid burning out. In my case, I try to balance it out for no more than once per month.
Here is my own ideal mix of the drive for learning. How about you?
The Channel: T shape
Reading in the most traditional way of learning from theories. In the modern world, watching a video and listen to podcast/audiobook are become viable, and more and more prominent, driving by the shorter attention span of the modern lifestyle. Classroom or organized training is the other formal way.
Each learning channel has its unique benefit. Backed by technology, listening to podcast or audiobook became more and more popular in recent years, and it will make your commute much more meaningful when stunk in traffic.
What “T” shape means is :
- Pick the main channel that is proven to be the most effective to you, and invest the majority the energy on it.
- Remain open-minded and leave a small portion of your available resources to other channels
This will help you avoid being a” jack of all trade and master of none”
For me is reading (books) and produce long-form content like writing blog, because I like to have the ability to see the big picture, and dive into detail when needed.
Resource: Money should not be your constraint
Think about managing yourself like a company: US public companies on record spent over 3% of their total revenue in “Research and Development”. Tech companies usually spent over 10%. Assuming you are making $70,000 a year, did you spend at $2,000 every year to improve yourself? If you are in tech, is $7,000 annual budget learning surprising you?
Furthermore, you are also likely to get sponsorship from your employer. A healthy company will support their employee to develop themselves, as it is a win-win situation for both parties.
Practice – “use it or you will lose it”
When I started my career a while ago, my boss used to tell me “In order to truly learn a skill, you really have to do the same thing over and over again, like 35-40 times, making it your second nature.”
Practice, or repetition with a purpose, transferred to knowledge from the book/podcast/course/words from others, to your brain. The process isn’t always fun – it takes time and willpower, both are more scarce than money.
That is also the reason that you can’t and won’t learn everything you wanted. Ideally, you will focus your energy to make consciously repetition to those things matter to you(self-fulfilling) and to the world(marketability).
Reflection – take the time to ask yourself, then adjust
Most of the time, practice isn’t enough. Reflection is the next step to help us move the needle. Other people like to call it “self-correction”, basically it means we need to be able to find out what works, what doesn’t work, and then make appropriate follow-ups.
Reflection happens unconsciously more often. For example, when I find myself struggling to find a large chunk of uninterrupted time for reading, naturally I started to buy more audiobooks. Therefore I can listen to the same content during my daily commute.
Conscious reflection can be real but still necessary. In the business world, the leadership usually set up strategy at the beginning of the year, and have multiple check-in points throughout the year to check-in and make adjustments. The same rule should apply to the individual development. Set up multiple reminders, and ask yourself: How did the learning process go? How can you make it better?
Community – one-man show isn’t sustainable
It takes a village to raise a child. The same rule applies to learning as well. Being part of the community with the shared goal of learning similar subjects has a series of benefits. This post listed my own experience in detail of how I was fighting the loneliness by actively participating community event.
However, it doesn’t need to be this formal. The function you’d like to develop might not have an active community in your local area. You can try the following two options:
- Start your own community: by simply finding another folk who shared the same interests as you, you can simply set up a regular check-in with this person to update each other what you have learned. In this way, you are holding each other accountable (ultimately holding yourself accountable) to learn the subject. The best way to learn is to teach others.
- Just Google it. Very likely there is a community out there virtually. Particularly If you are learning technical skills, the leading company usually maintain an online community, providing an environment for their “fans” to learn from each other. Example: SAP , PowerBI
This following video is inspiring to watch. I just don’t understand how he comes up with the magic number “20”.
Sleep tracking isn’t new. But what’s unique about using Apple Watch?
Sleep is the single most important activity for a normal human being, measured by time spent. Having a good night of sleep usually means a jump start of the day. On the flip side, bad sleep (or no sleep at all) will lead to serious implication – both physically and mentally.
In this blog post, I am happy to share my story of using the Apple Watch + 3rd Party App to better measure the sleep. Hopefully, it will inspire you to pick up a few things here and there and take action to make your own sleep better.
How everything started for me
First and most importantly, making the most out of my Apple Watch is fun. I just can’t wait to milk everything out of my $500 investment 18-months ago.
Also, I had sleep problems in the past. Anxiety and parenting are the two big drivers.
Lastly, professional curiosity also pushes me to find more meaningful data to analyze.
Why should you do it (benefit)
Similar to my previous post on tracking Tennis and Swim, having sleep metrics available will allow us to see our sleep through a new lens. “How you feel” after waking up is still an important signal, but no longer the only one.
What’s unique (compared to tracking tennis and swim) is, sleep tracking doesn’t require any extra button clicking. Just wearing the watch with you, making sure it has at least 20% battery, and the 3rd Party App (AutoSleep is my choice) will do the work in the background.
The biggest benefit is allowing you to ACTIVELY manage your own sleep: Once the habit is established – checking the sleep pattern the 1st thing in the morning. In addition, it can provide:
- Unbiased and more accurate protocol of sleep quality: Is qualitative sleep data (asleep time, deep sleep time, quality score, average heart rate) matching quantitative feeling like “Am I feeling refreshed”? If the answer is no, usually the quantitative data gives me a more unbiased indication. For example – when I wake up from a bad dream feeling awful but the sleep data tells me the sleep quality is actually okay, my daytime energy is generally fine.
- Give self an opportunity to reflect: Why today’s sleep is better or worse than yesterday? This usually leads to an honest conversation with self on “Is there anything I do before the sleep that I can relate to?” Over the time I have learned to avoid doing any work or exercise 2 hour before sleep. Ideally, screen time should be avoided as well – it is just too hard to make it a reality.
How you analyze the sleep(case study)
There are four metrics available from the home screen of the AutoSleep app.
- Sleep: the total hours of actually falling asleep recorded by the device.
- Quality: a calculated metric based on total sleep time, how restless and the nocturnal dip in heart rate and determines how much quality sleep you had
- Deep sleep: the number of hours where you met a sufficient nocturnal dip in heart rate combined with minimal movement.
- BPM: average heart rate during the sleep
After understanding the metric definition, we can easily export the data into a CSV file, dump it into Jupiter Notebook (Python) or R Studio(R) to wrangle, slice dice and visualize the data like crazy, and finally reach the following insights:
(I wish I can have the time to actually do the things above)
- Deep sleep happens early: Get to bed early and sleep equal or less time will improve our sleep. The concept might not be completely new, as research like this has made a similar point. I never truly believe it, until I see my own data. Below is a screenshot of a “typical night” (Technically more data point is needed than just one typical example). Being able to see my own sleep pattern is very powerful – the purple bars, indicating deep sleep, clutters almost exclusively before 3AM.
- High resting heart rate indicatesillness or vice versa: I don’t get sick very often. But when I am, my average heart rate went up significantly. Apparently, many others have realized the same thing. To be completely honest, this is a good-to-know metrics, and more likely we can only use it for “descriptive purpose” – a.k.a – “because of illness, the resting heart rate goes up. ” Personally, I am looking forward to the advancement of the technology, and we will be able to use it for prediction: “When our heart rate goes up by over 30%, the chances we are suffering from X disease is increasing by Y%”. Additional metrics, like blood pressure and body temperature will be needed. Will that happen in the near future? Let us wait and see.
Apple watch is not the only option in the market to track sleep. I have used Misfit in the past, and Fitbit is another popular option. Both of these two are cheaper than Apple Watch, and the features are very similar. However, those usually lack 3rd party analyzing tool.
Battery life: Putting on the watch when asleep means you won’t be able to charge it during the night. Usually, it will cost around 20% of the battery to track a normal night of sleep.
Comfort: It does feel weird at the beginning to wear a watch to sleep. It is better to turn on the “theater mode”, otherwise the watch screen might turn on accidentally during the night.
Price: Apple watch isn’t cheap. I won’t recommend to use it to track sleep as the main purpose. Starting with a Misfit Flash (cost less than $30) might be a more economical choice. However, if you already have one, the 3rd party app to track sleep is affordable: It cost less than $5.
I have recently wrapped up 5 years (4 years + 9 months to be exact) of analytics career in a Fortune 100 company. In this precious in-between moments, I’d like to take the time to reflect on 3 lessons learned as an analytics professional in the digital marketing industry.Read More »
You are a proficient Microsoft Excel user – vlookup is an appetizer, pivot-table is the main course, and VBA/Macro is your dessert. And your company just gave you an upgraded Microsoft 365 subscription, and a brand new version of Excel is just being served. You are wondering – “What’s new?”
As an analyst living and breathing with Excel, PowerPivot and PowerQuery are the two killer features, period. I’d like to share my story on how you can save time and make your spreadsheet work more efficient.Read More »
2017, I have been benefited greatly from the digital analytics association(DAA), including the eMetrics and certified web analyst destination, and five local chapter events in Toronto. This post breaks down these events I went to, what are they, how do they help me, and why they matter to me during the career development for a digital analyst.
Being a part of a bigger community
Earlier in my career, maybe 10 years ago, I had a similar feeling surrounded by a group of peers, sharing similar interests – tennis. We achieved great things together – workshops, tournaments, etc. That experience not only shaped my life-long hobby, but also gave me an opportunity to meet, work and have fun with a few friends that I am still contacting with now.
Back in 2017, I have a similar feeling – “being a part of something big”, after attending 5 local Toronto chapter events from DAA. Read More »
Back in February in my first post of using Apple Watch to track swim, I wrote about the metrics available and usage in the Apple Watch, a handful of data quality challenges, and my plan of using data captured to improve performance. After logging another 22 workouts since Mar 2017, I would like to give an update, and also share a few more new interesting lessons learned.Read More »
I have been using Microsoft product for over 20 years, starting with MS-DOS, and Windows 3.1. Microsoft PowerBI is my favorite software so far: It is simple, updated frequently, and has the right amount of configuration to “get things done”. The user community used to be small back in 2015, but it is catching up quickly with the forum, free resource (YouTube search “PowerBI”) and paid resource (high-quality material like SQLBI and PowerPivotPro) .
At the same time, as an analytics practitioner, I also understand it is vitally important to keep a mindset of “software independent”. What matters the most is the ability to deliver high-quality deliverable in the most efficient way. A technology tool like PowerBI can help to achieve that, however, we need to be cautious of falling in love with the software, not your own craft and the value delivered.Read More »
Sharing experience of preparing the certified web analyst exam offered by digital analytics association