Our previous blog post discussed the gap between data analysts and big data. In this post, we'll look at how those data analysts go about building models and leveraging big data to drive insights.
Market mix modeling is the science of establishing mathematical relationships between marketing tactics and sales. Statistical modeling is an important means to discover these relationships. Modeling isn’t new -- it’s been around for many decades.
From Excel to SAS, there are many tools an analyst can use to build models. However, it isn’t exactly a walk in the park to do marketing mix modeling. Why is it so difficult to put models to work for marketing analytics? For that, we have to first understand the many different steps before and after the model is built by the analyst:
1. The analyst loads the customer’s data into a database.
2. She cleans up the data by removing statistical anomalies.
3. She analyzes the data, perhaps running a few regressions.
4. She looks at the results, refining and augmenting the input. She might have to iterate over steps 3 and 4 multiple times. After multiple iterations, the analyst selects a system of equations that describes the relationships between marketing tactics and sales.
5. The data analyst deploys the model in a software package for the end user to build “what if” scenarios.
6. A solver might be deployed in an application to find optimized scenarios.
Now, the business user can provide new inputs about marketing activities. The software application gives the business user predictions about financial metrics or can even suggest improvements to the original inputs.
Simple enough, right? As they say in Inception, we must go deeper.
Lately, there has been a lot of online discussion and debate about the article “The Data Scientist Will Be Replaced By Tools.” While interesting and somewhat provocative, the piece takes some positions that we don’t agree with: namely, that due to a lack of talent in the data science field, new tools that allow for automation of data science, and new companies popping up that provide data science services, that the role of the data scientist will be diminished.
While we don’t believe that new tools and automation processes will replace the role of the data scientist/analyst (just like business intelligence tools did not replace the role of the business analyst), we do think that there is a significant gap between the data analyst and today’s rapidly evolving area of Big Data.
The data analyst extracts meaning from data using modeling, equations and optimization. But in order to access the data they rely on a computer analyst who responds with phrases like MapReduce, HDFS, NoSQL and Amazon Web Services. How do you bridge this gap?Add a comment