52% of Organizations Say Risk Factors are a Critical Consideration When Evaluating AI Use Cases
“An AI-first strategy is a hallmark of AI maturity and a driver of increased return on investment,” said Erick Brethenoux, Distinguished VP Analyst at Gartner. “However, AI-first does not mean AI-only. While AI-mature organizations are more likely to consider AI for every possible use case, they are also more likely to weigh risk as a critical factor when determining whether to move forward.”
The survey was conducted in October through December 2022 among 622 respondents from organizations in the U.S., France, the U.K. and Germany that have deployed AI. Gartner defines an “AI-mature” organization as those who have deployed more than five AI use cases across several business units and processes, in production for more than three years.
Across all organizations, respondents had deployed an average of 41 AI use cases, with use cases remaining in production for 3.5 years (see Fig.1)
Fig. 1: Average Duration of AI Use Cases in Production
Source: Gartner (July 2023)
AI-Mature Organizations Involve Legal Counsel at Ideation
The most significant differentiator identified among AI-mature organizations was the involvement of legal counsel at the ideation stage of AI use cases. AI-mature organizations were 3.8 times more likely to involve legal experts at the ideation phase of an AI project’s life cycle.
“There is uncertainty around the ethics and legality of various AI tactics, as well as a fear of violating privacy regulations,” said Brethenoux. “Organizations that are more experienced with AI do not want to be told they’ve crossed a line once they are further along in the process of developing an AI use case.”
AI-Mature Organizations Evaluate Technical and Business ROI Metrics
When evaluating the return on AI investment, 52% of AI-mature organizations focus on a combination of technical and business metrics to assess ROI. In less mature organizations, technical metrics are most often used to measure the value of AI use cases.
More AI-mature organizations – 41% compared with 24% of all others – use customer success-related business metrics to estimate ROI. Furthermore, 47% of AI-mature organizations cite customer service as one of the top three business functions benefiting from AI, compared with 34% of others.
“Many business and IT leaders focus on AI’s impact on optimization and productivity, but organizations do not prosper through cost cutting,” said Brethenoux. “Organizations that are using AI technology to attract and retain customers are able to more clearly articulate the impact on the business, driving a virtuous cycle of executive buy-in for new AI projects.”
AI-mature organizations are also more likely to define metrics earlier in the AI lifecycle. Sixty-seven percent of AI-mature organizations define metrics at the ideation phase of every use case, compared with 44% of less mature organizations.
