Data-Driven Decision Making: The Foundation of AI Innovation

Almost all of the recent innovations in AI are due to the ability to amass large quantities of data. The discipline of Machine Learning takes this data and outputs insights never available before. AI can accurately identify images and individual faces, it can process and translate text, and can reliably respond to speech. The companies who have access to the most data, in the proper form, are able to build the most powerful AI applications. Think of Amazon who process millions of transactions, Google who process millions of searches, and Facebook who process millions of posts and pictures each day. All of this is data and is captured in a format that can be successfully fed into algorithms which then provide the insights and outputs valuable to businesses. As companies attempt to catch up to the big players, they need to avoid mindless approaches to data collection and ensure they build robust data generating processes.

Companies have been collecting data for decades, if not centuries. Data-driven companies have the ability to move quickly and make informed decisions to drive profit. The problem is that companies who run legacy IT systems are not collecting data in digital form, nor in the quantities needed, and not in the format required to successfully train Machine Learning algorithms. This leaves management in a position where they need to ask a few key questions:

1. Should we go through a digital transformation?
2. What data is essential to our decision making and AI applications?

The ability to capture insights, get to know your clients better, drive down costs and make better decisions are things that almost all companies can benefit from. This makes the answer to the first question simple for those who wish to maintain market share as competitors with digital capabilities enter the market. The second question leads management to an analysis of what data to capture and what items to focus on while going digital.

The time preceding a digital transformation is when management really needs to fully analyze the current business model, understand use cases for AI and other technologies, and see where this transformation will take them. Earlier we noted that many companies have been collecting data long before the establishment of the internet. It would be foolish to not take a step back and determine which data collection processes are essential, and which are outdated and providing no value. Charlie Munger of Berkshire Hathaway tells a story of how poor data collection can lead to poor decision making:

The water system of California was designed looking at a fairly short period of weather history. If they'd been willing to take less perfect records and look an extra hundred years back, they'd have seen that they weren't designing it right to handle drought conditions which were entirely likely. You see again and again- that people have some information they can count well and they have other information much harder to count. So they make the decision based only on what they can count well. And they ignore much more important information because its quality in terms of numeracy is less- even though it's very important in terms of reaching the right cognitive result. All I can tell you is that around Wesco and Berkshire, we try not to be like that. We have Lord Keynes' attitude, which Warren quotes all the time: "We'd rather be roughly right than precisely wrong." In other words, if something is terribly important, we'll guess at it rather than just make our judgement based on what happens to be easily countable.

Even though Charlie is speaking about a data collection process that may not have been applying Machine learning techniques, his statements remain true today. Management can fool themselves by making decisions on available information rather than the most important information. In the new world of AI, management has an incredible opportunity to rethink the way they collect their data and utilize it to make decisions. They mustn't squander this opportunity by doing what is easy, only collecting small sample sizes, or simply duplicating old data collection processes. Those who take the time to reflect on the past will be better prepared for the future.