AI, particularly generative AI and large language models (LLMs), has made tremendous technical strides and is reaching the inflection point of widespread industry adoption. Companies across various sectors understand the need to embrace the latest AI technologies or risk being left behind. However, the field of AI safety still poses a significant challenge. The risks associated with AI and machine learning going rogue are well-documented, with examples of hidden biases in algorithms and unintended consequences causing reputational damage to companies.
Microsoft’s Tay Chatbot is perhaps the best-known cautionary tale for corporates:
Trained to speak in conversational teenage patois before being retrained by internet trolls to spew unfiltered racist misogynist bile, it was quickly taken down by the embarrassed tech titan — but not before the reputational damage was done.
Even highly acclaimed AI models like ChatGPT have been criticized for their shortcomings. With the potential of AI revolutionizing industries, corporate leaders and boards recognize the need to navigate the minefield of AI safety concerns.
The answer lies in focusing on a class use cases I call a “Needle in a Haystack” problem.
Haystack problems are ones where searching for or generating potential solutions is relatively difficult for a human, but verifying possible solutions is relatively easy.
Identifying Haystack Problems: Initial Use Cases
- Checking documents for spelling and grammar mistakes: While computers can catch spelling mistakes, gen AI can accurately identify grammar mistakes, making it easier for humans to verify and correct them.
- Generating boilerplate code from unfamiliar APIs or libraries: AI trained on collective code written by software engineers can automate the process of generating boilerplate code, saving time and effort in coding.
- Processing scientific literature: AI can assist scientists in processing and assimilating the vast amount of scientific knowledge available, helping identify interdisciplinary insights and accelerate discoveries.
The Importance of Human Verification in AI
The critical insight in all these use cases is that while solutions may be AI-generated, they are always human-verified. Directly allowing AI to make decisions or take actions on behalf of a company poses significant risks. Human verification of AI-generated content is crucial for AI safety.
Having a human verify the output of AI-generated content is crucial for AI safety.
Focusing on Haystack problems improves the cost-benefit analysis of that human verification.
This lets the AI focus on solving problems that are hard for humans while preserving critical decision-making and double-checking for human operators.
By focusing on Haystack use cases, companies can gain AI experience while mitigating potential safety concerns. With the right approach, AI can unlock its revolutionary potential while ensuring human oversight and safety.
Tianhui Michael Li is president at Pragmatic Institute and the founder and president of The Data Incubator, a data science training and placement firm.
