Big Data

The Death of Why- Part II

Bill Schmarzo By Bill Schmarzo CTO, Dell EMC Services (aka “Dean of Big Data”) December 19, 2011

My previous blog, “The Death of Why – Part I,” discussed how big data analytics is negating the need to understand “why” things happen.  With the vast amounts of detailed data available and high-powered analytic tools, it is possible to identify what works without having to worry about why it worked.  This can enable us to “think different” about the user experience and the context in which we present business insights to our users.

A New Analytics Approach

The capabilities of big data analytics to identify significant, material, and actionable insights buried in the data, without having to understand why these insight occur, provides the foundation for an entirely different user experience.  Instead of presenting an endless series of reports and dashboards in hopes that users can discover what’s important to their business, we can instead reverse the traditional analytic process and present to the users only that which is important to their business.  We do this by starting with the interesting insights, and then provide the relevant data, reports, and dashboards required to support those insights (see chart below).

User Experience Ramifications

The user experience can be greatly simplified to present only the information or insights needed to optimize the business.  The user experience could focus on two types of insights:

  • Observations are “unusual” behaviors or performance (e.g., 2 standard deviations outside normal performance, 200% above/below predicted performance) that might require further investigation by the user.  Observations would leverage simple analytic models to identify performance situations operating outside of normal boundaries.  These would be potential starting points for a more detailed investigation by the user. For example:
    • Did you know that product A’s sales to the [Soccer Dads] customer segment is 150% of what it was last period?
    • Did you know that marketing campaign [2011 Back To School Sale] is 50% below forecasted conversions with only 2 weeks left in the campaign?
    • Did you know that the variance level on machine [Charles City Plant Turbine #120] is 20% outside the normal control boundaries?
    • Recommendations are specific actions that are generated based upon a detailed analysis of the current state of the business.  Recommendations would leverage advanced analytic modeling and real-time feeds to score performance, analyze the key business drivers and variables, update models, and make specific recommendations.  For example:
      • We recommend marking down the product category [Christmas Lights] by 25% starting December 9, and increasing the markdown to 50% on December 16.
      • We recommend increasing the media budget by 22% on display ad [Chevy Suburban] and decrease media budget 33% on display ad [Chevy Volt] for the remainder of the campaign [Holiday Season].
      • We recommend repairing your [Maytag Model 3200] washer’s drum engine within the next 5 days because there is a 95% probability of product failure.

The user experience could start with a series of Observations and Recommendations, priority ranked by their potential impact on the business. If the user then wanted more details on the Observations and Recommendations, they would select the [More] button to get the supporting details (see example below using the iPhone as the delivery platform).


The analytics that underpin the Observations and Recommendations could be quite complex.  The analytics could be personalized and self-learning so as to be constantly fine-tuning the analytic models based upon the users’ feedback on what they like, didn’t like, and why (think Pandora).

Massive, detailed data sources, coupled with more powerful analytic tools, provides the capabilities to identify significant, material, and actionable insights in the data without forcing users to have the analytic skills or training to quantify why things happened.  It allows us to provide a completely different user experience – one that is focused on providing greatly simplified Observations and Recommendations – to help the business users optimize their key business initiatives.

Bill Schmarzo

About Bill Schmarzo

CTO, Dell EMC Services (aka “Dean of Big Data”)

Bill Schmarzo, author of “Big Data: Understanding How Data Powers Big Business” and “Big Data MBA: Driving Business Strategies with Data Science”, is responsible for setting strategy and defining the Big Data service offerings for Dell EMC’s Big Data Practice. As a CTO within Dell EMC’s 2,000+ person consulting organization, he works with organizations to identify where and how to start their big data journeys. He’s written white papers, is an avid blogger and is a frequent speaker on the use of Big Data and data science to power an organization’s key business initiatives. He is a University of San Francisco School of Management (SOM) Executive Fellow where he teaches the “Big Data MBA” course. Bill also just completed a research paper on “Determining The Economic Value of Data”. Onalytica recently ranked Bill as #4 Big Data Influencer worldwide.

Bill has over three decades of experience in data warehousing, BI and analytics. Bill authored the Vision Workshop methodology that links an organization’s strategic business initiatives with their supporting data and analytic requirements. Bill serves on the City of San Jose’s Technology Innovation Board, and on the faculties of The Data Warehouse Institute and Strata.

Previously, Bill was vice president of Analytics at Yahoo where he was responsible for the development of Yahoo’s Advertiser and Website analytics products, including the delivery of “actionable insights” through a holistic user experience. Before that, Bill oversaw the Analytic Applications business unit at Business Objects, including the development, marketing and sales of their industry-defining analytic applications.

Bill holds a Masters Business Administration from University of Iowa and a Bachelor of Science degree in Mathematics, Computer Science and Business Administration from Coe College.

Read More

Share this Story
Join the Conversation

Our Team becomes stronger with every person who adds to the conversation. So please join the conversation. Comment on our posts and share!

Leave a Reply

Your email address will not be published. Required fields are marked *

0 thoughts on “The Death of Why- Part II

  1. Hi Bill,

    The ‘why’ still plays an important part here, but you seem to have relegated it to untested assumptions. That doesn’t sound like progress to me.

    The recommendations here assume an underlying cause of the observations. Without understanding the probable reason ‘why’ the observations occurred, you risk taking the wrong action.

    Marking down the product assumes that reducing the price 25% will result in an optimal profit, when 12% might have been the correct amount. Or something else other than price is causing the problem, and you’ve just sacrificed margin on a low-margin product category with no idea why you’re doing it.

    Data analytics provides an opportunity to discover more ‘why’, not less. And then, sure, you can offer simplified solutions, like “97% of the time, when the power variance on the turbine has this pattern, it signals a breakdown within 2 months, so you should get the bearings serviced. It’s cheaper.”

  2. Thanks Justin for your comments. The challenge that I see as I talk to business users is that they don’t have the skills or time to go through a detailed analysis to understand “why” something happened, especially on the day-to-day decisions upon which they are confronted. Granted, a more detailed analysis may give the user a better answer (e.g., 12% price reduction may have yielded a better profit), but at what cost? It takes time and resources to do a more detailed analysis, and that likely means that other decisions get delayed, or missed entirely.

    There are certain types of decisions (strategy, acquisitions, patient care) where a more thorough and disciplined analytics process is warranted, and the time and resources are likely available to make that more thorough analysis. But many, many day-to-day decisions do not warrant that level of analysis. Business users would be elated to have the application give them guidance, suggestions and recommendations on how to improve performance on those types of daily decisions.

    Thanks again Justin. Good observation and question.

  3. Not having the time or skill to understand your own business is a worry. And if you’re wasting time on things that aren’t important to your business, well, we need to have a chat about that. 🙂

    I think I’m mostly agreeing with you, Bill. I can certainly see value in outsourcing the “why” part to someone more skilled for simple, common issues. I don’t care why a disk drive thinks it’ll fail soon, I just let the engineers come and replace them.

    But the detailed analysis has to happen somewhere. In the disk drive case, someone figured out what disk failure looks like, and what’s a good predictor. Now we have a pretty good rule for “if X, then Y”, and it frees up my time for other things. That’s great!

    In my experience, getting people to actually take good advice is the tricky part. There’s lots of good information out there (in books!) on how to do pricing, motivate employees, provide good customer service etc., and plenty of companies who do the exact opposite. And actively resist anyone advising them of better ways to do things.

    Maybe people will listen to a computer more than a human. I guess we’ll find out.

  4. Hi guys, think your debate point is over “who cares about the “why””? As a consumer, I don’t care why my washing machine has failed, I just need it fixed. The engineer cares, because they want to prevent it, but as a customer, I have no intention of funding that insight for the benefit of someone else.

  5. What I am seeing from many customers is the need for a fast decision, more than the need for the perfect decision. Many customers are seeing opportunities die before they can act upon them. The pricing example is a good one, because retailers (both online and brick and mortar retailers) are faced with various different pricing scenarios on a multitude of products on a daily basis. Most do not have the capabilities (technology-wise or personnel-wise) to try to make the perfect pricing decisions in all cases, especially given the constantly changing variables (economic, weather, competitive, social, local) that impact the decision. So in these cases, some decision – even a suboptimal decision – is better than no decision.

    Also, it is important to put into place an analytics system that learns from the previous pricing decisions; that is constantly testing, experimenting and refining its analytic models. So while the initial 12% decision might not have been optimal, eventually the system works towards a “more optimal” decision.