Busted! A MythBusters Approach to Big Data Analytics
During a Big Data analytics project you uncover a ton of information about your business. All of it expands your knowledge, and some of it leads you to new and interesting hypotheses. The trick is to quickly and iteratively test those hypotheses to get the most value out of them – because sometimes they’re not going to lead anywhere. We recently tested out a few using our EMC Customer Service data and I want to share our approach with you.
Step 1: Have Fun!
We decided to use MythBusters as our theme for executive read-outs. As silly as this seems, it lightens the mood and gets people more relaxed. Sometimes advanced data analytics can be very complex and your point can get missed. Find a way to simplify your message and make it memorable.
Step 2: Assemble Your Myth-Busting Team
Read my last post for how to get started here.
Step 3: Use A Repeatable Analytics Process
As we build out our Big Data analytics capabilities for EMC Global Services, we have been documenting the approach we have taken. We’ve also referenced material from EMC Education Services training courses on Data Science to help us on the way:
Step 4: Share Your Insights
Don’t wait for perfection – you may find out quickly that your hypothesis is busted. Set the expectation with your leadership that not every hypothesis will be confirmed and let them know when one is….
- Busted: The data doesn’t suggest your hypothesis is correct. Or, the business identifies reasons why the results of the hypothesis test are wrong or misleading. Human-created data is full of problems, so be prepared for this one if you are using it. Exploring eventually-busted hypotheses is not wasted time. Learning what “is not” is key.
- Plausible: The data suggests the hypothesis is possible and the business hasn’t found any issues with your initial analysis. This will often lead to further analysis or more questions to be answered. If you are struggling with the “define the hypothesis” phase of your projects, this step can feed future hypotheses.
- Confirmed: The data suggests the hypothesis is possible and the business confirmed that to be the case. I don’t see this outcome coming too often in the area of predictive analytics. There is always some level of error or further exploration…If you found a model that is 100% perfect please call me. : )
Focus your energy on time-to-insight, then share your findings in a fun and consumable way! This is an iterative approach and the sooner you get feedback, the sooner you can add value and/or adjust your project. Otherwise you could spend a lot of time and money and still get busted.