Why Industry — Not Technical — Expertise is Key to Fixing Your AI

When artificial intelligence first went mainstream, companies of all kinds jumped at the technological gold rush. All of a sudden, PhDs in machine learning, natural language processing and other forms of AI, whose value largely remained confined to the walls of academia, were being plucked out of universities and offered jobs at some of the top tech companies. Likewise, engineering became the most popular college major for incoming high school graduates. But successful AI projects require more than just writing code. 

When AI fails, it usually isn’t because of a lack of engineering talent, but a lack of training. AI is designed to provide an answer based off of thousands, and sometimes millions, of data inputs. The algorithms interpret and group unlabeled data according to similarities. But an input can sometimes be muddled, which can lead to an incorrect output or decision. 

Despite this, companies look to engineers and data scientists to quickly deploy AI — sometimes without proper planning for how they’ll maintain and train the AI. The content and feedback signal that feeds the system needs to be updated regularly and this maintenance requirement is a huge hurdle, which is how most companies end up with useless AI. Training over time validates or invalidates those decisions, and the algorithms continually learn to make better decisions. 

In the rush to deploy AI, many companies saw AI technology as a quick upgrade to their offerings to keep up with competition. However, it takes a village to create intelligent AI at scale. A 2018 report on digital transformation revealed that most companies are not prepared to effectively use the technology they have invested in — including AI. In fact, Gartner estimates that AI failure rates will reach as high as 85 percent by 2022. 

It’s one thing to get an AI initiative up and running; it’s another to devote the time and money to actually make it smart. The real intelligence in AI comes from training AI with accurate data from the people who understand the material most: subject-matter experts.

Subject-matter experts are those who know the ins and outs of a topic. In business, they are your customers and super fans. They are the ones using your product or service every day. Usually, they are already providing feedback either directly or indirectly — on forums, FAQs, Twitter, and by simply using the product. They usually know its pain points more intimately than the people who built it. By incorporating their expertise into your systems, you can create a better customer experience.

An AI left to its own devices will collapse over time. With continuous human feedback, also known as “human-in-the-loop” AI, the AI gets smarter. By putting humans at the center of the system, AI can consistently provide the correct output. But we can take this one step further: “Expert-in-the-loop” AI incentivizes a company’s network of expert users to train AI at scale.

Cumulatively, your customers have a wealth of information about your products that has barely been tapped. And this collection of expertise is the key to making your AI useful. In the customer service industry, subject-matter experts are typically better equipped to train an AI-powered virtual agent than an internal support team. They use the product and understand the concerns other customers have better than anyone. 

For example, Samsung won a Stevie Award for sales and customer service after the company integrated Directly across 75 million devices. The AI project helped Samsung achieve 95 percent customer satisfaction. Impressively, Directly resolved 80 percent of questions without requiring agents. By incorporating subject-matter expertise into your systems, you not only create a better product, but you also create opportunity in the AI era for other types of specialists outside of those with traditional analytical or technical skills.

While some forms of AI don’t require expert skills to train algorithms — such as image recognition, which merely requires people to identify objects — more nuanced applications of AI will need subject-matter experts to train the systems accurately. For example, in customer service, a company can employ its expert customers to create content and train AI to resolve support issues by better understanding customer questions and surfacing the right content.

Not everyone can be a computer scientist, and contrary to popular belief, we don’t need everyone to be. We need people who are experts in their own domain to help launch and train AI-based technology. To create a better AI product, engineers, algorithms, and lines of code alone will not get you there. We need to be asking the people who use the product to train the system — and reward them for doing so. They don’t need a doctorate to join. They just need to bring something they already have: subject-matter expertise.

NOTE: This piece was originally published in CIO Review.