You Need an Algorithm, Not a Data Scientist

[From my post on Harvard Business Review]

Mark Twain once said: “The past does not repeat itself, but it rhymes.” Although future events have unique circumstances, they typically follow familiar past patterns. Today, data scientists can predict everything from disease outbreaks to mortality to riots.

It’s no surprise, then, that companies trying to hear the rhymes and see the patterns in their sales conversions are trying to manually analyze their own data, hire the best data scientists, and train their managers to be more quantitative.

However, this people-centric, high-touch approach is not scalable. Markets are too dynamic, and some of the changes too imperceptible, to be realistically captured by humans.

Consider a company that is selling electronic devices. Let’s say that historically they have been selling well to companies that value their fast delivery and the quality of their product. As time passes, the competition grows and a global trend for green products arises. The profile of the company’s perfect customer slowly shifts and could go unnoticed by manually examining the market. However, those small shifts are identifiable by algorithms that continuously monitor the historical sales cycle of the company, cross-referencing it with external sources, like social media posts and newspaper articles discussing these trends, and finding correlations with the propensity to buy. Due to the size of this information base and its unstructured nature, monitoring all those delicate changes in real time becomes an almost impossible task for a human analyst.

While few companies have the luxury of having data scientists with the expertise needed to develop these sophisticated algorithms, nor the staff to analyze the results effectively, there is less need today. Data science today requires fewer experts, as many more automated tools are being developed and used to analyze thousands of events. (Disclosure: my company, SalesPredict, is in this industry.) The more sophisticated tools require very little or no human intervention, zero integration time, and almost no need for service to re-tune the predictive model as dynamics change.

Today, automated algorithms can identify patterns and provide insights such as:

  • Did you notice a big portion of your customer churn is from companies who have not used one specific feature of your product in the last three months?
  • Did you notice that the leads that converted to closed deals this month were from medium size high-growth companies who were searching for keywords comparing your product to your competitor?

But as your business changes, the answers will change as well, requiring more and more automation to track those changes and supply the business leader with real-time, actionable recommendations that are always relevant.

In the next few years, I believe many businesses, especially B2B, will use prediction in their business. But those who get the most from these analytics will be those that use automated algorithms – which are faster, more accurate, more scalable, and more adaptive than manually analyzed data.

In stock trading, human analysts once did the trading. Today, more and more automated machine learning algorithms accompany their decisions. It has become much harder to compete without such algorithms. Similarly, in the next few years, very few businesses can afford not to have automated decision making systems mining their data and suggesting the best next actions – not only in Operations, but in the Marketing, Sales, and Customer Success departments too. Following a large amount of ever-changing information will be the competitive edge.

One thought on “You Need an Algorithm, Not a Data Scientist

  1. We need more and more new data and good optimization for a simple algorithm.
    So much Big Datas, so much possibility, we need a new technology to analyse and optimize 1 at 10 Trillions of Gigabits per second. Kira Radinsky give us the possibility to think what we can do better, with a good optimization of the big database, today. The most important is: The date, the time, and the event and you have story (bad or good). We work to do it better. Kira Radinsky, please don’t close your future. You do a very good job.So, Kira your success is our success.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s