Changing architectures
Just how to best eliminate that overhead remains a topic of continual discussion and many divergent opinions. That's because despite years of evolution, BI is less a set-and-forget proposition than a conceptual framework for improving corporate intelligence.
That framework has evolved rapidly as BI tools--executive information systems, as they were called a decade ago--grew in sophistication and became accessible to all, with the rise of the standards-based intranet.
Yet data management remains a complex task: data warehouses were once seen as the best way of aggregating data from what was often dozens of disparate databases, but they also imposed a need for onerous data cleansing exercises and put a severe strain on server resources.
Data marts, the compromise developed to reduce the sheer volumes of data available for analysis by distributing relevant subsets of the data warehouse to individual departments, added additional complexity by adding data currency issues to the mix. But years later it is OLAP (online analytical processing), whose multidimensional "cubes" allow users to slice data subsets in a variety of ways, that has become the major driver for analysis of BI data.
With today's servers powerful enough to handle the massive data analysis projects their forebears could barely contemplate, OLAP provides an excellent way of letting users manipulate data as they see fit-as long as they know what they're trying to ask.
"It's always been a back-end issue as to whether the database could handle multiple queries," says Hummingbird Australia technical manager Paul Segal.
"Throwing around large volumes of data isn't so much an issue anymore. The biggest challenge is understanding the data in the database; if you don't know what the data is trying to tell you, you can't form any business decisions based on that data. Designing OLAP cubes requires a different mindset to normal reporting, but once people see its power they don't want to go back."
Segal's comments are reflected in IDC's analysis of the BI market, which predicted OLAP will be its second fastest growing segment, with 22 percent annual growth indicating widespread acceptance of its approach to data analysis. Indeed, OLAP revenues are expected to account for 27 percent of the overall BI market by 2005.
Interestingly, data mining--using artificial intelligence to pick out previously unforeseen patterns in data--is nowhere near the giant it was once was; its extreme complexity is part of the reason companies now design BI with specific business questions in mind.
Not everyone finds OLAP to be a breeze, points out Alphablox founder and chairman Michael Skok. "The average [OLAP] cube has six or seven dimensions. How many users can think beyond three? If your job is to slice and dice data all day, these are fantastic products; the question is how to get those people into the business."
There's little surprise, then, that IDC named the fastest-growing segment of the BI market as end-user query and reporting tools, which will grow at 30 percent annually to account for 52 percent of the BI market by 2005.
Such tools are critical in delivering the benefits of data analysis to users across the enterprise. Ease of use is a key factor these days: if normal users can't access the technology, it's simply of no use to them.
"A lot of businesses have done a lot for the deployment of data management, improving data quality, and have put BI tools in place," he explains. "The biggest challenge is delivering that efficiently to the Web, so they can get as high as possible into the organisation and talk about the total organisational view rather than getting stuck in a departmental view. The portal does that."
The ability to aggregate multiple data sources was a key driver for the Royal Australian Navy's decision in May to purchase up to 1000 seats worth of Hummingbird's BI/Broker business intelligence portal solution, to improve employee access to the systems used for administering its Material Systems Branch. The OLAP-based system will be used to generate over 100 reports a day on everything from product inventories to maintenance schedules.
Ubiquitous access to BI can present its own problems, however, and it's important to manage OLAP proliferation carefully, points out SAS Institute solutions manager James Redman. "Because the newer tools fit on anyone's desktop and everyone can build their own cubes, the proliferation of independent views of things has grown rather than receded," he explains.
"We advocate having a central warehouse with a single version of the truth; the problem is that warehouses take a long time to build and a lot of users don't want to wait that long. Six months down the road everyone's got their own little islands of information."
That can lead to many different perspectives of the same information, which in turn can cause friction between business units if managers and other users aren't well trained about the contextual meaning behind their numbers.
