Big Data

Leveraging Advanced Analytics to Power Digital Transformation

Bill Schmarzo By Bill Schmarzo August 21, 2017

“You can’t stop the incessant march of economics” – Bill Schmarzo

Okay, so it’s probably not cool to quote oneself, but hey, this is my blog and I get to do what I want.  And for anyone who follows me knows, I love to “riff” on the game-changing power of economics.  The “economics of big data” – where the cost to store, manage and analyze data is 20x to 100x cheaper than traditional analytics – started this big data and data science craze.  But ultimately it is the economics of value, or to be specific, “value in use” where the economics really become a game changer.

I recent article titled “The Simple Economics of Machine Intelligence” from the Harvard Business Review highlights very well the role that economics (maybe even more than data science) is going to play in separating the winners from the losers in digital transformation.  To quote the article:

  • Machine intelligence is, in its essence, a prediction technology, so the economic shift will center on a drop in the cost of prediction.
  • When the cost of any input falls so precipitously, there are two other well-established economic implications. First, we will start using prediction to perform tasks where we previously didn’t. Second, the value of other things that complement prediction will rise.
  • Using the language of economics, judgment is a complement to prediction and therefore when the cost of prediction falls demand for judgment rises.

So how will the “economics of machine learning” – or the “economics of advanced analytics” – impact your business model?  We developed the Big Data Business Model Maturity Index as a framework help organizations understand where and how they can leverage data and analytics to power their business models (see Figure 1).

Figure 1:  Big Data Business Model Maturity Index

Figure 1:  Big Data Business Model Maturity Index


The Maturity Index provides a roadmap to guide customers in integration data and analytics into their business models.  See “Big Data Business Model Maturity Index Guide” for a “How To” guide on leveraging data and analytics to advance along the Maturity Index.

Advanced Analytics Continuum

Recent conversations with Walker Stemple of Intel’s @intelAI organization got me thinking about where and how organizations can leverage “advanced analytics” to power their business models.  Now “advanced analytics” is a broad definition, but I have included the following analytics in that definition: Regression, Clustering, Neural Networks, Machine Learning, Deep Learning, Artificial Intelligence and Cognitive Computing.  And while these “classifications” seem to change on a regular basis (sometimes due to us getting smarter; sometimes due to non-value-add marketing hype), it is critical that tomorrow’s business leaders understand where and how to apply these advanced analytics to power their business models.

My conversation with Walker helped me to understand how organizations can leverage the clusters of advanced analytics to advance along the Maturity Index (see Figure 2).

Figure 2:  Advanced Analytics Continuum

Figure 2:  Advanced Analytics Continuum


The Advanced Analytics Continuum covers the following classifications:

  • Descriptive Analytics is not really advanced analytics, but it is foundational in helping organizations understand “What happened?” to their business. This is traditionally the domain of Business Intelligence.  Business Intelligence is primarily focused on “Comparative Analytics” such as Current Period versus Previous Period reporting, Period-to-date cumulative calculations and projecting trend plots.  The primary analytic tools in Descriptive Analytics are reports, dashboards and alerts.
  • Predictive Analytics is focused on uncovering insights about what happened in order to create foresight, or predictions, about what is likely to happen. Predictive Analytics seek to quantify cause-and-effect – and measure the analytic model’s goodness-of-fit – in order to drive those predictions.  Predictive analytic algorithms include Statistics, Clustering, Classification, and Regression Analysis.  This is also where you will see the use of ANOVA, Covariance and Confusion Tables to measure the goodness of model fit.
  • Prescriptive Analytics is focused on building analytic models that can prescribe or recommend what actions consumers and employees should take in order to optimize key business or operational processes. Advanced analytic algorithms of choice in the area of Prescriptive Analytics include Collaborative Filtering, Neural Networks, Deep Learning, and Machine Learning.
  • Cognitive Analytics is focused on creating an environment (system or application) that can self-monitor, self-diagnose, self-fix and ultimately self-learn. These environments are constantly measuring the effectiveness of decisions and updating/refining the analytic models based upon the outcomes of decisions.  Think about an autonomous vehicle that is moving through a new environment and has to learn quickly about the nuances of that environment (e.g., traffic patterns, potholes, road maintenance, temperature variations, wind gusts, precipitation). Advanced analytics tools of choice in the area of Cognitive Analytics include Reinforcement Learning, Artificial Intelligence, and Cognitive Computing.

We can bring this all together by showing how the Advanced Analytics Continuum can help organizations advance along the Big Data Business Model Maturity Index as seen in Figure 3.

Figure 3:  Advanced Analytics Power Business Model Maturity Index

Figure 3:  Advanced Analytics Power Business Model Maturity Index


While not perfect (and likely will never be perfect) the merging of the Advanced Analytics Continuum with the Big Data Business Model Maturity Index helps organizations to understand not only in what advanced analytics to invest, but understand how those analytics can help advance the organization along the maturity index.

Advanced Analytics in Action

Let’s walk through an example of how we might apply the Advance Analytics Continuum to create an intelligent organization.  For our example, I am using publicly available data from the City of San Jose Open Data Portal ( that shows where and when fatal traffic accidents have occurred in the San Jose Area.  The City of San Jose Open Data Portal is part of the city’s Open Data Community Architecture (ODCA) initiative led by the City of San Jose Data Architect, Arti Tangri and CIO, Rob Lloyd.  The ODCA project provides a highly adaptable reference architecture that encourages local community analytics by providing an adoptable design that (1) exposes data for use, (2) arranges analytics and skills for informed decisions, (3) builds a platform that enables automation and prediction, and (4) aims the data for shared data lakes for government/academia/private sector use.  In these structures, the ODCA lays the foundation for economic monetization of the City of San Jose’s data.

Descriptive Analytics:  Reporting Fatal Accidents

Let’s start the advanced analytics transformation process by creating descriptive analytics about what has happened.  For example, let’s say we have a map showing fatal accidents over a select period of time (see Figure X).

Figure 4:  Reporting of Fatal Traffic Accidents

Figure 4:  Reporting of Fatal Traffic Accidents


While Business Intelligence is a great starting point, we must embrace advanced analytics to become more actionable.

Predictive Analytics:  Predicting Where Accidents Are Likely To Happen

We can apply Predictive Analytics (e.g., Statistics, Clustering, Classification, Regression Analysis) to “quantify cause-and-effect” in order to predict when and where a fatal traffic accident is likely to occur (see Figure 5).

Figure 5: Predicted Fatal Traffic Accidents

Figure 5: Predicted Fatal Traffic Accidents


In order to create this prediction, we need to brainstorm with the key stakeholders the variables and metrics that might help us make a better prediction of fatal traffic accidents.  In particular, we want to brainstorm the following question:

What data might you want in order to predict when and where a fatal traffic accident might occur in the South Bay?

The brainstorming exercise will produce a wide variety of data sources that the data science team might want to consider as they build the predictive analytics (see Table 1).

Traffic Patterns Weather Conditions Local Events
Time of Year Day of Week Time of Day
Holidays Population Growth Economic Activity Growth
Building Permits Road Maintenance New Driver Permits
Tourism Large Truck Traffic Electric Vehicles
Changes in Retail Sales Building Renovations Pedestrian Traffic
Bike Traffic Motorcycle Traffic Sunrise Location

Table 1:  Potential Data Sources for Predicting Fatal Traffic Accidents

Prescriptive Analytics:  Recommending Where and When to Locate Police

Next we want to leverage prescriptive analytics in order to augment human decision-making and optimize key business and operational processes.  In our reduce crime application, Prescriptive Analytics can create recommendations about when and where to locate police (see Figure 6).

Figure 6: Recommended Police and Emergency Equipment Locations

Figure 6: Recommended Police and Emergency Equipment Locations

It is important that the application captures the actual decisions made about where and when the police and emergency equipment are located in order to measure the effectiveness of the recommendations.  This feedback on the decisions and the associated outcomes is critical in creating Cognitive Analytics.

Creating a Self-learning, Intelligent Accident Response App

Finally, we want to leverage Cognitive Analytics to create an intelligent or learning application to reduce fatal traffic accidents.  We want the application to continuously learn from new environments and potentially new data, more granular data sources (see Figure 7).

Figure 7:  Cognitive Analytics to Continuously Learn and Refine Analytic Outcomes

Figure 7:  Cognitive Analytics to Continuously Learn and Refine Analytic Outcomes


The model should tell the data scientist which variables are the most predictive (variable predictability or the relative importance of a particular variable to the analytic model’s results) so that proper effort can be placed in ensuring that the data is complete, accurate, timely and governed.

Summary: The Economics of Advanced Analytics

The process of determining or quantifying the economic value of an organization’s data based upon Adam Smith’s “value in use” concept (“Wealth of Nations”, 1776) is greatly augmented via advanced analytics; where “value in use” is defined as the use of an asset to create new economic or financial value (see the University of San Francisco research paper “Applying Economic Concepts To Big Data To Determine The Financial Value Of The Organization’s Data And Analytics Research Paper” for more details on how to determine the economic value of your organization’s data).  Advanced Analytics drive the “value in use” aspects of economics by creating new digital assets (data, analytics and intelligent applications) that power an organization’s business models and fuel an organization’s digital transformation.

Tomorrow’s business leaders cannot be content to leave the understanding of advanced analytics to just their data science or business analyst teams. Tomorrow’s business leaders must be at the forefront of understanding the capabilities of advanced analytics so they can determine where and how to apply advanced analytics to power today’s as well as tomorrow’s business model battles.

In the end, it’s not having the advanced analytics capabilities that will determine the winners from the losers, but it’s where and how organizations exploit advanced analytics to re-invent their business models.

Bill Schmarzo

About Bill Schmarzo

Read More

Share this Story
Join the Conversation

Our Team becomes stronger with every person who adds to the conversation. So please join the conversation. Comment on our posts and share!

Leave a Reply

Your email address will not be published. Required fields are marked *

2 thoughts on “Leveraging Advanced Analytics to Power Digital Transformation