With all the generative AI hype swirling among evangelists, one might think that the Fortune 500 is galloping wildly towards putting large language models (LLMs) into production and turning corporate America into one big chatbot. To that, I say: “Whoa, Nelly!” — meaning, think again. That’s because for all the C-suite executives out there feeling generative AI FOMO and getting pressure from CEOs to move quickly to develop AI-centric strategies, things are actually moving far slower than you might imagine (or AI vendors, who warn companies about falling behind, might want).
As I reported back in April, there’s certainly no doubt that executives want to access the power of generative AI, as tools such as ChatGPT continue to spark the public imagination. But a KPMG study of U.S. executives that month found that a solid majority (60%) of respondents said that while they expect generative AI to have enormous long-term impact, they are still a year or two away from implementing their first solution.
The Deliberate Approach to Adoption
“We’re all anxious to see results right away… [But] we need to feel comfortable about the accuracy… to ensure the risks are well managed.”
Consider Marco Argenti, CIO at Goldman Sachs — who told me in a recent interview that the leading global investment banking, securities and investment management firm has, nearly a year after ChatGPT was released, put exactly zero generative AI use cases into production. Instead, the company is “deeply into experimentation” and has a “high bar” of expectation before deployment. Certainly this is a highly-regulated company, so careful deployment must always be the norm. But Goldman Sachs is also far from new to implementing AI-driven tools — but is still treading slowly and carefully.
While Argenti told me that he thinks “We’re all anxious to see results right away” in areas like developer and operational productivity, as well as revolutionizing the way knowledge workers work and producing content, when I asked him what it would take to put its experimental use cases of generative AI into production, he said it required “feeling comfortable about the accuracy.” He added that this needs to hit a certain threshold “in which we feel comfortable that the information is correct and the risks are actually well managed.” In addition, he said that Goldman Sachs needs a clear expectation of a return on investment before deploying generative AI into production.
The Importance of Accuracy and ROI
“Everybody’s trying to seek confirmation that there is usefulness, or really trying to kind of see the ROI for these investments.”
One use case that is clearly showing “huge” progress, he explained, is software development, where he said the company is seeing 20-40% productivity gains in its experiments and has a target of getting 1,000 developers fully equipped to use generative AI tools by the end of the year. Argenti emphasized that Goldman Sachs is not simply randomly running AI models. From the very beginning, he said that the company has implemented a platform that ensures technical, legal, and compliance checks. The front-end server has measures in place to filter out any inappropriate content, while all interactions are logged to ensure the data used is fully authorized.
Goldman Sachs want to be methodical and thoughtful, he said, because “it’s very easy to get carried away.” Argenti recalled a recent dinner with over 30 CEOs at large companies, in which he warned against a hyper-focus on productivity enhancement with generative AI. “That will not cause differentiation, sooner or later everyone will have it… it will establish a new baseline on productivity,” he said. “It’s also trying to find cycles and courage of investing in things that might not be profitable today, that are more about how is our business going to change? What’s the new role of an advisor, what’s the new role of an investor, or a trader? We’ve been very careful in trying to strike that balance in a way that we’re still very conscious of the fact that this could actually not just be sustaining technology but also disruptive technology.”
But while Goldman Sachs may not exactly be galloping at top speed into the Wild West of generative AI, Argenti clearly maintained that the company won’t be falling behind, either. “We have a lot of horses in the race,” he said. “So we feel pretty good about that.”
