Why Machine Learning and Why Now?
Machine learning is no longer just for smartphones or game shows. Here’s how to develop a strategy that will change the basis of competition in your industry.
by Daniel Wellers, Jeff Woods, Dirk Jendroska, and Christopher Koch
When the Netflix series House of Cards premiered in 2013, it quickly became the most downloaded content in the company’s history – a statistic that came as no surprise to Netflix executives. They had previously examined a vast pool of Netflix data on subscribers’ viewing habits and determined that the show was likely to become a hit even before they purchased it.
The wisdom behind Netflix’s sure-fire choice came from machine learning, which, loosely defined, is the ability of computers to learn on their own (without being programmed) by using algorithms that churn through large quantities of data.
Machine learning’s talents aren’t limited to picking the next TV blockbuster, either. Consider some of the more down-to-earth uses that we already take for granted today. Have you noticed how spam e-mails have almost disappeared from your inbox? That’s machine learning. Or how you can casually converse with anthropomorphic voices coming from your smartphone? Also machine learning.
But these examples pale when compared to machine learning’s potential for remaking business. Increased data-processing power, the availability of Big Data, the Internet of Things, and improvements in algorithms are converging to power a renaissance in business intelligence.
The untapped potential of machine learning
Here are some ways that machine learning could transform the core elements of the business ecosystem– and society:
Intelligent business processes. Many of today’s business processes are governed by rigid, software based rules. This rules-based approach is limited in its ability to tackle complex processes. Further, these processes often require employees to spend time on boring, highly repetitive work, such as checking invoices and travel expenses for accuracy or going through hundreds or thousands of résumés to fill a position. If we change the rules and let self-learning algorithms loose on the data, machine learning could reveal valuable new patterns and solutions that we never knew existed. Meanwhile, employees could be reassigned to more engaging and strategic tasks.
Intelligent infrastructure. Our economy depends on infrastructure, including energy, logistics, and IT, as well as on services that support society, such as education and healthcare. But we seem to have reached an efficiency plateau in these areas. Machine learning has the potential to discover new signals in the data that could allow for continuous improvement of complex and fast-changing systems. That gives humans more time to apply their creativity (something that machines may never learn to duplicate) to new discoveries and innovation.
Digital assistants and bots. Recent advances in machine learning technology suggest a future in which robots, machines, and devices running on self-learning algorithms will operate much more independently than they do now. They may come to their own conclusions within certain parameters, adapt their behavior to different situations, and interact with humans much more closely. Our devices – already able to react to our voices – will become more interactive, continuously learning assistants to help us with our daily business routines, such as scheduling meetings, translating documents, or analyzing text and data.
Plan for change
Although machine learning has already matured to the point where it should be a vital part of organizations’ strategic planning, several factors could limit its progress if leaders don’t plan carefully. These limitations include the quality of data, the abilities of human programmers, and cultural resistance to new ways of working with machines. However, the question is when, not if, today’s data analysis methods become quaint relics of earlier times. This is why organizations must begin experimenting with machine learning now and take the necessary steps to prepare for its widespread use over the coming years.
What is driving this inexorable march toward a world that was largely constrained to cheesy sci-fi novels just a few decades ago? Advances in artificial intelligence, of which machine learning is a subset, have a lot to do with it. AI is based on the idea that even if machines can’t (yet) duplicate the actual structures and thought patterns of the human brain itself, they can at least offer a rough approximation of important functions, such as learning, reasoning, and problem solving.
AI has been around since the 1950s, but it didn’t take off until the late 1990s, when Moore’s Law’s true exponential effects on computing power were realized, and researchers reined in their impulses to build a mechanized brain, focusing instead on using algorithms and machine learning to solve specific problems. Highly publicized machine-learning triumphs by IBM, such as Watson’s drubbing of human contestants on Jeopardy, captured the imagination of the public and business leaders.
Machine learning comes in several flavors, sometimes referred to as supervised learning (the algorithm is trained using examples where the input data and the correct answers are known), unsupervised learning (the algorithm must discover patterns in the data on its own), and reinforced learning (the algorithm is rewarded or penalized for the actions it takes based on trial and error). In each case, the machine can learn from data – both structured (such as data in fields in a spreadsheet or database) and, increasingly, unstructured (such as e-mails or social media posts) – without explicitly being programmed to do so, absorbing new behaviors and functions over time.
Machines’ ability to learn puts them on an evolutionary path not unlike our own. They are gaining the ability to speak, listen, see, read, understand, and interact with ever-increasing sophistication. In just the last four years, the error rate in machine-learning–driven image recognition, for example, has fallen dramatically to near zero– practically to human performance levels.
Machine learning as collaborator
As machine-learning–based skills approach those of human beings, it’s tempting to view their evolution as a zero-sum competition with humans that we are destined to lose.
However, there is another view that says that automation will lead more to collaboration rather than outright replacement. Consulting firm McKinsey & Company argues that while 49% of jobs will be subject to some degree of automation, just 5% will be fully replaced anytime soon. In most cases, says McKinsey, automation will take over specific tasks rather than entire jobs.
McKinsey’s argument is compelling, at least when it comes to knowledge work, because it mirrors the way computing has evolved within the organization. Early mainframes were programmed to perform specific tasks, such as tallying up an organization’s daily receipts. When PCs were first introduced in the 1980s, they were dismissed by businesses as expensive typewriters until packaged spreadsheet software came along, allowing organizations to automate some of their manual accounting tasks at the individual employee level. Knowledge work would never be the same.
Today, most organizations have enterprise software that uses rules-based processing to automate many tasks in departments such as finance and human resources and in warehouses. Yet while the task-based automation of enterprise software has brought tremendous productivity improvements, the software could not learn and improve with experience as humans can.
Thanks to advances in computer processing power, memory, storage, and data tools, machine learning can be integrated into the enterprise-software systems that form the heart of most organizational IT infrastructures. This means that the software, using the mastery that it develops in individual tasks, will be able to contribute increasing levels of performance and productivity to the organization over time, rather than merely offering a one-time boost, as most software packages do today.
The strength of machine-learning integration
The improvements the software brings to organizations will not be limited to individual tasks. One of the biggest strengths of enterprise software is its integration– the ability of individual applications to share information and be part of process workflows both within individual departments and across the organization. Integration allows organizations to experiment with new combinations of ever-more intelligent and versatile machine-learning applications and, where possible, let the machines learn how to improve the ways they work with each other and with their human colleagues. Together, these applications form the intelligent enterprise.
Just as individual applications will contribute more productivity to the organization as their embedded machine-learning abilities become more sophisticated, so too will the combinations of those applications evolve to bring more intelligence and flexibility to departmental and organizational processes over time.
Here are some concrete examples of how machine learning is creating value in organizations today:
A machine-learning system can be trained to recognize payments that arrive without an order number and match them to invoices based on knowledge of customers’ order and payment histories. This lets organizations reduce the amount of work outsourced to service centers and frees up finance staff to focus on more strategic tasks.
A machine-learning system can learn to pluck the most suitable job candidates from the thousands of résumés that organizations receive. It can also spot biased language in job descriptions that might discourage qualified people from applying and rescue other top candidates who fall through the cracks because they don’t fit with traditional hiring models.
By building models based on historical transactions, social network information, and other external sources of data, machine-learning algorithms can use pattern recognition to automatically spot anomalies. This identification helps detect and prevent fraudulent transactions in real time, even for previously unknown types of fraud. And this type of algorithmic security is applicable to a wide range of other situations, including computer hacking and cybersecurity.
Let’s say that your flight is suddenly delayed. A travel app on your smartphone can use context-sensitive machine learning to determine how this delay will affect your other travel plans and prompt you with rescheduling options.
of a pipeline, for example, and automatically spot areas that need maintenance or replacement.
Machine learning needs a platform
To be sure, organizations will gain tremendous benefits from individual machine-learning applications, even if they are never integrated into a larger whole. However, the benefits become much greater when these applications are on an integrated platform.
The business press has been discussing the power of platforms a lot lately, with iTunes being a well-known example. By creating a set of common software development tools that are available free to anyone who wants them, Apple has enabled developers to create thousands of applications for the iTunes App Store. Developers win because they can easily reach vast numbers of Apple device owners through iTunes. Apple wins because it takes a cut of the revenues for each app it makes available in the App Store.
Platforms are equally important to enterprises, not necessarily because of the profit motive (though some organizations are launching their own public, for-profit platforms similar to iTunes), but because having a platform gives them a base for quickly and cost-effectively combining different applications together, whether they are from different software vendors or are built in house.
No software vendor will ever be able to claim that it offers every machine-learning–enabled application that an organization needs out of the box. But vendors do offer platforms that organizations can use as bases for building out their entire machine-learning infrastructure.
The core of these machine-learning–enabled platforms is application programming interfaces (APIs). APIs are a kind of software version of those universal electric plug adapters that business travelers lug around with them so they can charge their electronic devices wherever they may be in the world. APIs allow software developers to plug into another software vendor’s applications without having to know anything about the complex code at the heart of those applications.
Another benefit of having a unified software platform is that organizations can use it to create a single point of access to data from across the organization. Data is the sole nutrient in a machine-learning diet. Algorithms need to binge on it constantly to lead a healthy and successful life. The larger and richer the data set, the more accurate the results. Having a single platform helps break down the data silos that exist across the organization so that organizations can make the most of machine-learning intelligence.
Organizations don’t need to go it alone
Inevitably, organizations will want to develop machine-learning–based applications that are not available in the marketplace. However, this does not mean that they need to create large internal machine-learning centers of expertise (although having some internal experts is recommended). Service providers can bring the expertise and perspective from within and across industries to help organizations focus on a small set of highly strategic processes that will benefit from machine learning.
The first step toward developing such applications is to determine where to apply machine learning. Organizations need to ensure that it erects barriers to entry against competitors or provides new ways of capturing and retaining customers by improving repurchase cycles or achieving new levels of win rates.
That means focusing investments on the machine-learning problems that will matter most to the industry’s basic competitive economics. Developing those engines will take considerable effort and time, so focusing the enterprise on those one or two projects that will really make a difference matters.
Here are five criteria to determine how to apply machine learning in a way that will create lasting differentiation.
1. The focus area as an appropriate candidate.
Not every facet of business will benefit from machine learning. The greatest potential is in automating high-volume tasks that have complex rules and large amounts of unstructured data.
Is your focus area big and complex enough for machine learning?
2. A clearly formulated issue. Machine learning works best on specific, well-defined tasks where the desired output and relevant inputs can be clearly stated: given X, predict Y. While it isn’t a magic bullet that will automatically help organizations learn from all the data in their enterprise, machine learning can be valuable in discovering correlations in large amounts of data that humans could never have deduced for themselves.
3. A sufficient quantity of examples to learn from. Machine learning requires a lot of data to be accurate. There must be enough examples for the machine to learn meaningful approximations of the decisions you want to make. This is discovered through experimentation.
4. Meaningful differences within the dataset. If the data you are trying to learn from does not contain meaningful differences, then the algorithm will fail at its mission. Let’s say that you are trying to identify different types of buyers. If the training data does not contain significant differences in buyer characteristics, the algorithm cannot give you useful results.
5. A clear definition of success. Machine learning is always evaluated by measures of performance on a specific task. Typically, the computer will try to optimize whatever performance measure is defined. Clear evaluation criteria for the algorithm are therefore critical. You also need to be certain that the evaluation criteria are actually helpful for solving your business problem.
Key evaluation criteria for machine learning
The human factor
Ultimately, the technical barriers to machine-learning adoption will be easier to solve than the human ones. Predictions of steep job losses due to automation are stoking fear and uncertainty about how these self-learning systems will impact our roles and our livelihoods.
These fears must be addressed, and significant investment must be made in change management as business processes and models are reworked to integrate self-learning systems into collaborative human-machine environments.
Indeed, self-learning machines have the potential to become valuable collaborators with humans, augmenting their skills and helping employees become more productive in their current jobs while freeing them from boring, repetitive tasks.
Experts also predict that machine learning will create new roles inside the organization. There is already a shortage of data analysts and those capable of developing the intricate algorithms that machine learning requires. Other new roles will become evident as machine learning integrates deeper into the organization – and not all roles will require a degree in computer science or math. For example, creative thinking, strategy development, quality management, and people development and coaching will be crucial skills in an AI-driven organization, according to a survey by consulting firm Accenture2.
When machine learning matures to the point that it can handle unstructured data (still an issue today), when organizations openly share data, and when algorithms begin to interact with each other more freely, machine learning will be embedded in all systems, devices, machines, and software. That will enable highly context-sensitive insight at both the organizational and individual levels. We can only guess at the level of automation that will result, but the impact on business – and society – will be significant.
Already, commercial machine-learning applications based on these technologies are available, and more are being created all the time. That is why business leaders should engage now with trusted providers that can help them evaluate data structures and availability, free up information from siloed systems, and identify the richest areas for machine-fueled insight and improvement. Together, they can address the cultural and change management challenges to take advantage of this new wave of business intelligence.
Download the white paper Why Machine Learning and Why Now?