Big Data

Big Data 101 for Investment Management

By Dave Bagatelle March 13, 2013

To many, the term “Big Data” is hard to define.  Is it just more data in a standard relational database or is it something else?

The next question usually is… How can it help me?

Let me start by comparing it to the standard relational database approach.  In this architecture source files are imported through an ETL (extract/transform/load) process into a data warehouse where the information is then accessible through various applications.

For an investment management firm, a simple example would be client account data being combined with holdings and transaction data and then exported to a monthly statement.

Big Data is something else and does not replace the standard relational database model.

Big Data streamlines the process and removes the need for the ETL process and speeds the loading of data to end applications by splitting and then re-combining data when requested by the end applications.  This simplistic explanation may not be clear, but what you need to know is that it is fast and can quickly combine both structure and unstructured data.

Let’s get to the more interesting topic… How it can help

Investment Managers that use trading models know that their value comes from three main areas:  data, how it is being analyzed, and the speed at which it is processed.  The flexibility of these models is also critical and it is always better to reduce development time (and cost).

All of these make trading models a perfect fit for Big Data solutions.


Solutions (meaning, the front and back-end infrastructure) that can analyze both structured and unstructured data and be integrated to automatically execute and report (ex. pre- and post-trade risk engines and management and client reporting tools, etc…) are immensely valuable to the investment management industry.


A system that can quickly analyze large volumes of both historical and real-time data enables analysts and traders to find market inefficiencies and ultimately alpha for their clients.


Complex trading strategies can be tested and revised at significantly less cost than previously possible giving investment managers an edge over retail and institutional investors that have not invested in the tools and back-end infrastructure required to be competitive in the current market.


Investment management firms all value security expertise and many are understanding the value of developers and quants (data scientists), but just as important is the technology infrastructure.

Big Data solutions are relatively new and exciting technology.  I urge firms to assess their technology to gain (or maintain) a competitive advantage.

About Dave Bagatelle

David Bagatelle, CFA is an Advisory Consultant for the Asset and Wealth Management division of EMC Consulting as part of Global Services.

David offers a unique blend of financial services experience combined with technical knowledge and project management experience and has provided both strategic advice (roadmaps, vendor selection) and tactical implementation services to an array of large financial services firms. He specializes in financial planning, client reporting, performance measurement, account aggregation, attribution, analytics and advises on operational strategy for the front middle and back offices of institutional clients.

Prior to joining EMC, David worked at multiple money managers and advised clients on managing their personal finances. David holds a BBA from the John M. Olin School of Business at Washington University in St. Louis and is a CFA charterholder.

Read More

Join the Conversation

Our Team becomes stronger with every person who adds to the conversation. So please join the conversation. Comment on our posts and share!

Leave a Reply

Your email address will not be published. Required fields are marked *

0 thoughts on “Big Data 101 for Investment Management

  1. Today, I attended Big Data Symposium at Palo Alto, CA. It was hosted by EMC. We learned a lot about how storage/mpp ( massively parallel processing) could fit in bigger big data plays. This article touches those plays plus general Hadoop. Upcoming Pivotal Hadoop Distribution from EMC is positioned to achieve big data results, combining best of MPP and Hadoop HDFS.