Big Data

Big Data Dashboards…

Bill Schmarzo By Bill Schmarzo CTO, Dell EMC Services (aka “Dean of Big Data”) January 24, 2012

Given my Business Intelligence (BI) background (3 years at Business Objects as the head of their Analytic Applications business unit), I’m naturally drawn to conversations about how the world of big data is going to impact the BI world.  Now, the BI world has been taking quite a beating lately as the excitement surrounding the new data scientist role sweeps the information management industry.  But it would be foolish, if not naïve, to sweep aside all the work that the BI specialists have put into codifying organizations’ key performance indicators into interactive dashboards and reporting environments.  So instead of talking about replacing yesterday’s BI efforts, let’s brainstorm how we can leverage these new big data trends to super-charge our BI investments.

Today’s Management Dashboards

The BI professionals have done an outstanding job of working with their business constituents to define the metrics and key performance indicators against which the business will monitor and manage the business.  They’ve invested the time and the effort to create the reports, dashboards and scorecards to help the business better manage their businesses.

To quote my friend Dr. Pedro DeSouza, we can integrate advanced analytics with our existing dashboards and reports to give context to an event. We can use descriptive statistics to give simple, direct context to an event by positioning the event against the average and standard deviations calculated from history. Then we can use predictive statistics to forecast the likelihood of situations and other events for a given context. For example, Mondays after the Super Bowls (which won’t include the San Francisco 49’ers this year) are the days with the highest incidence of absenteeism: lots of “sick calls.” We can predict how many people will call sick and even predict who will call sick given the person’s attributes.

Let’s take a look at a sample dashboard.  For this example, I have developed a simple dashboard that could be used by a package delivery territory manager to better manage their daily package delivery operations.

In this sample dashboard, the Delivery Operations Manager is interested in monitoring the following 4 key performance indicators:

  • District Daily Deliveries Performance –used by the Ops Manager to compare the performance of their district to the previous periods and district averages.   The Ops Manager will use this to identify unfavorable trends that might require further investigation.
  • Average Minutes Spent per Driver –is used to measure the effectiveness of the drivers and the route planning algorithms and to see if there might need to be some changes to improve individual driver delivery performance.
  • Driver Absenteeism –used to identify and track trends and patterns in driver absenteeism, which is critical to proactively identify individual driver performance and absenteeism trends.
  • Successful versus Unsuccessful Deliveries –is used to track the percentage and cost of missed deliveries, and costs associated with missed deliveries.  Missed deliveries are probably the most important preventable cost item under the control of the Ops Manager.

This dashboard helps the Ops Manager monitor the key performance indicators that dictate territory performance and identify areas of the business that might require further investigation and investments.  And each of the graphics in the dashboard would be interactive in that it would allow the Ops Manager (and their staff) to drill into more detail across multiple dimensions (e.g., days, drivers, routes, trucks).

Tomorrow’s Real-time, Predictive Dashboards

So how do we expand upon this current BI and dashboard investment?  We’ll explore adding two new characteristics or dimensions to the dashboards:

  1. Make the dashboard more real-time (or lower-latency) with respect to shrinking the time between when the data event occurs and when the data is available for analysis by the Ops Manager
  2. Add more predictive capabilities (especially with respect to treating the existing KPIs as dependent variables) to provider finer-fidelity reporting, insights and possibly recommendations.

Let’s look how we could modify the existing dashboard KPIs and graphics (see graphic below):

  • In the Daily Deliveries Performance analysis, we could move from a daily update of the delivery performance metrics to a minute-by-minute data feed.  The benefit here is that problems could be identified and resolved within the same day that the problem was identified (where additional costs could be avoided), instead of having to wait until the next day when it is too late to fix that problem.
  • In the Average Minutes per Delivery analysis, we could add more variables to the analysis, including detailed weather conditions and traffic data, in order to create finer-fidelity models that can predict average delivery times more accurately.  And these models could be refined throughout the day as weather and traffic conditions change so that the average delivery times could be updated and communicated more frequently throughout the day.
  • In the Driver Absenteeism analysis, we could again add more variables to create a more accurate Driver Absenteeism model.  Factors such as hunting season and NFL football games could be taken into consideration, as well as unstructured social media data that might provide insights into other driver “distractions” that might impact absenteeism.  Individual models could be built at the driver level that incorporate all of these new data feeds, yielding even more accurate absentee forecasts and predictions.
  • Finally, the coup de grace, predicting and minimizing costs associated with Successful vs. Unsuccessful Deliveries analysis.  Here, we might want to build a more comprehensive predictive model based on the improvements in the other KPIs (average minutes to delivery, absenteeism score, real-time delivery performance) to create intra-day predictions of unsuccessful deliveries.  This would arm the Ops Manager with the insights necessary to take intra-day actions to more quickly identify and resolve unsuccessful delivery problems (and save the company substantial money).

Let’s take a more detailed look at how we could use a combination of predictive analytics and low-latency data feedback to more closely monitor intra-day delivery performance.

The chart above is an updated version of the Daily Deliveries Performance analysis. I included a real-time chart showing the delivery performance of a day against the average plus and minus one sigma (standard deviation).  In this example, the Ops Manager can see right away that the day didn’t start very well, with the aggregated performance slightly above “Average –1 Sigma”, which is still within tolerance. However, the situation deteriorates during the morning and, by lunch, the aggregated performance of the day is below one sigma of the average. The manager could then take action at that moment to root cause the problem and remediate the situation.



I find the potential for super-charging companies BI investments very exciting, maybe because I was one who over the past several years helped companies create these analytic BI and dashboard environments.  Now with many of these big data innovations (e.g., social media data, high velocity data feeds, MPP architectures, in-database analytics, in-memory computing), we have the opportunity to build upon that BI investment and create a business environment that is both more real-time and more predictive.


By the way, I will be speaking at the Strata Conference in Santa Clara, CA.  I will be speaking as part of Strata’s JumpStart MBA program on Tuesday, February 28th (Happy Birthday Alec!) from 2:20 to 2:40.  The title of my session is entitled “Do it Right – Proven Techniques for Exploiting Big Data Analytics.” I will also be conducting “Office Hours” on Thursday, March 1st at 2:20 (whatever office hours is supposed to mean).  If you’re at the conference, please stop by and say hey!!


Bill Schmarzo

About Bill Schmarzo

CTO, Dell EMC Services (aka “Dean of Big Data”)

Bill Schmarzo, author of “Big Data: Understanding How Data Powers Big Business” and “Big Data MBA: Driving Business Strategies with Data Science”, is responsible for setting strategy and defining the Big Data service offerings for Dell EMC’s Big Data Practice. As a CTO within Dell EMC’s 2,000+ person consulting organization, he works with organizations to identify where and how to start their big data journeys. He’s written white papers, is an avid blogger and is a frequent speaker on the use of Big Data and data science to power an organization’s key business initiatives. He is a University of San Francisco School of Management (SOM) Executive Fellow where he teaches the “Big Data MBA” course. Bill also just completed a research paper on “Determining The Economic Value of Data”. Onalytica recently ranked Bill as #4 Big Data Influencer worldwide.

Bill has over three decades of experience in data warehousing, BI and analytics. Bill authored the Vision Workshop methodology that links an organization’s strategic business initiatives with their supporting data and analytic requirements. Bill serves on the City of San Jose’s Technology Innovation Board, and on the faculties of The Data Warehouse Institute and Strata.

Previously, Bill was vice president of Analytics at Yahoo where he was responsible for the development of Yahoo’s Advertiser and Website analytics products, including the delivery of “actionable insights” through a holistic user experience. Before that, Bill oversaw the Analytic Applications business unit at Business Objects, including the development, marketing and sales of their industry-defining analytic applications.

Bill holds a Masters Business Administration from University of Iowa and a Bachelor of Science degree in Mathematics, Computer Science and Business Administration from Coe College.

Read More

Share this Story
Join the Conversation

Our Team becomes stronger with every person who adds to the conversation. So please join the conversation. Comment on our posts and share!

Leave a Reply

Your email address will not be published. Required fields are marked *

0 thoughts on “Big Data Dashboards…

  1. Bill

    This is a great post- on a related note how do you see the traditional BI center of excellence in contrast to the data scientist role?

    Do you think that folks in a COE can move into a Data Scientist role? Would be interesting to hear what you’re seeing in that regard.

  2. Hey Sameer, thanks for your question. Given my recent opportunities to get “deep dipped” into the world of data scientists, I do think that BI folks are well positioned to make the transition to Data Scientists (or to add Data Science to their BI repertoire). The BI professionals will have to learn new tools, techniques and processes, but they’ll be able to build upon their BI skills, number 1 of which is a solid understanding of the metrics, KPI’s, data, question, decisions and processes that LOB users use today to manage their businesses. I believe that most BI professionals would find the addition of Data Science skills to be refreshing – and that the opportunity to help transform their businesses from a retrospective way of monitoring the business to a more predictive, real-time way of optimizing the business a lot of fun!

  3. I am not easily impressed, especially in Data Visualization. However, I am blown away by this presentation. Simply awesome and inspiring. Thanks for sharing.

  4. Bill,

    Donald M here, long time no speak, hope all is well.

    As you might expect I have a view on this!

    I completely agree with the improvements you outline above they are all valid and useful. In fact they are possibly the first “big data, predictive” suggestions for a dashboard I have seem which make sense.

    However, there is a (much?) bigger impact that dashboards can have on the world if big data. Namely getting into the hands of more users.

    As you will be painfully aware, BI adoption has been stuck at about 20% for years. IMO the reason for this is that we insist on force feeding analyst tools to end users when what they need is something which is better suited to their needs, i.e. an interactive dashboard(for a longer discussion in eBook at

    The current hype around big data and predictive threaten to make this worse. Both predictive and big data are incredibly important technologies but if we insist (as many do) on simply throwing them at end-users with a data-discovery veneer then they will waste the opportunity.

    By contrast interactive dashboards are a near-perfect interface for an end-user to access a big data set, and tellingly an good interactive dashboard on a mobile device is pretty much indistinguishable from a normal mobile app (which is the way of the BI future).

    So how about an example of a big data, predictive dashboard than anyone can use with no training :-

    It might not be what you expected, but IMO it is the reference point for the future of big data BI. Deliver BI like this (or like an iPad app) and suddenly user adoption limits will be a thing of the past. Lots more details on why i think this at in a series of articles summarized here

    Would love to catch up at some time to talk about all of this in more detail.