Turning Failure into the Foundation for AI Success
A recent report by the Rand Corporation suggests that approximately 80% of AI projects in businesses fail. While this statistic presents a daunting picture, it's essential to recognize that failure is a critical component of innovation, especially when it comes to emerging technologies such as AI.
The narrative that billions of dollars are wasted on AI projects fails to capture the experimentation and learning that drives technological progress. If a project fails but provides valuable insights into what works and what doesn't, it isn't a waste. Instead, it’s a step forward. The idea that innovation can proceed without encountering failure is not only unrealistic but counterproductive. With each failure, we move closer to understanding the technology and how to deploy it successfully.
Interestingly, the failure rate for AI projects today is an improvement from years past. Previously, it was estimated that 87% of AI projects never made it out of the proof-of-concept stage. This improvement reflects our growing competence in managing AI’s complexities.
So why the improvement?
For one, AI systems of the past required a lot of structured data management. Today’s AI systems still require data but in a form that looks more like natural language—documents, emails, and other text-based data. Both developers and businesspeople have a better handle on these information sources compared to the rigid data structures inside relational databases or JSON.
This evolution in data requirements has made AI development more intuitive and accessible. But as always with new technology, we experience an initial wave of excitement followed by a phase of disillusionment. That's where we are now—the "bashing and hitting" phase. People question the technology's efficacy, pointing to its high failure rate as a reason to doubt its potential. Being upset by this misses a fundamental point: failure is a necessary part of the path to success. To understand use cases where AI is appropriate and the tasks within an organization to which it can be effectively applied, firms have to endure a learning curve laden with failed experiments.
We need to embrace failure as an integral part of the innovation process. If we’re to navigate the complexities of AI and harness its full potential, we must be prepared to fail—again and again—until we succeed. In doing so, we will eventually reach a level of understanding that allows us to deploy AI in ways that are both effective and transformative.
Kristian Hammond
Bill and Cathy Osborn Professor of Computer Science
Director of the Center for Advancing Safety of Machine Intelligence (CASMI)
Director of the Master of Science in Artificial Intelligence (MSAI) Program