A few years ago, the business technology world’s favourite buzzword was ‘Big Data’ – a reference to organisations’ mass collection of information that could be used to suggest previously unexplored ways of operating, and float ideas about what strategies they may best pursue.
What’s becoming increasingly apparent is that the problems companies faced in using Big Data to their advantage still remain, and it’s a new technology – AI – that’s making those problems rise once again to the surface. Without tackling the problems that beset Big Data, AI implementations will continue to fail.
So what are the issues stopping AI deliver on its promises?
The vast majority of problems stem from the data resources themselves. To understand the issue, consider the following sources of information used in a very average working day.
In a small-to-medium sized business:
Spreadsheets, stored on users’ laptops, in Google Sheets, Office 365 cloud.The customer relationship manager (CRM) platform.Email exchanges between colleagues, customers, suppliers.Word documents, PDFs, web forms.Messaging apps.
In an enterprise business:
All of the above, plus,Enterprise resource planning (ERP) systems.Real-time data feeds.Data lakes.Disparate databases behind multiple point-products.
It’s worth noting that the simple list above isn’t comprehensive, and nor is it intended to be. What it demonstrates is that in just five lines, there are around a dozen places where information can be found. What Big Data needed (perhaps still needs) and what AI projects also rest on, is somehow bringing all those elements together in such a way that a computer algorithm can make sense of it.
Marketing behemoth Gartner’s hype cycle for artificial intelligence, 2024, placed AI-Ready Data on the upward curve of the hype cycle, estimating it would be 2-5 years before it reached the ‘plateau of productivity’. Given that AI systems mine and extract data, most organisations – save those of the very largest size – don’t have the foundations on which to build, and may not have AI assistance in the endeavour for another 1-4 years.
The underlying problem for AI implementation is the same as dogged Big Data innovations as they, in the past, made their way through the hype cycle – from innovation trigger, peak of inflated expectations, trough of disillusionment, slope of enlightenment, to plateau of productivity – data comes in many forms; it can be inconsistent; perhaps it adheres to different standards; it may be inaccurate or biased; it could be highly sensitive information, or old and therefore irrelevant.
Transforming data so it’s AI-ready remains a process that’s as relevant today (perhaps more so) than it’s ever been. Those companies wanting to get a jump start could experiment with the many data treatment platforms currently available, and as is becoming the common advice, might begin with discrete projects as test-beds to assess the effectiveness of emerging technologies.
The advantage of the latest data preparation and assembly systems is that they are designed to prepare an organisation’s information resources in ways that are designed for the data to be used by AI value-creation platforms. They can offer, for example, carefully-coded guardrails that will help ensure data compliance, and protect users from accessing biased or commercially-sensitive information.
But the challenge of producing coherent, safe, and well-formulated data resources remains an ongoing issue. As organisations gain more data in their everyday operations, compiling up-to-date data resources on which to draw is a constant process. Where big data could be considered a static asset, data for AI ingestion has to be prepared and treated in as close to real-time as possible.
The situation therefore remains a three-way balance between opportunity, risk, and cost. Never before has the choice of vendor or platform been so crucial to the modern business.
(Source: “Inside the business school” by Darien and Neil is licensed under CC BY-NC 2.0.)
Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and co-located with other leading technology events. Click here for more information.
AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.
