The most obvious ongoing trend in storage is simple: size. Storage capacity, whether it be in disk, tape, optical or other formats, is steadily increasing, and the cost per megabyte continues to fall.

Don’t be fooled, however, into thinking this means that storage is getting cheaper. “The vast majority of the cost in storage is not in the disk but in the personnel,” says Simon Johnson, director of enterprise solutions at Dell. The problem is exacerbated by businesses which choose the seemingly “cheap and quick now” option of simply adding extra servers when storage demands increase.

“They have farms and farms of servers to manage, and it’s not a very elegant solution,” says Harry Christian, regional director for Network Appliance. “They’ve grown on the back of storage needs, rather than using servers to deliver applications.”

The trend is understandable. “Many applications need lots of temporary space, so people tend to overprovision, and you end up with excess,” says Graham Titterington, senior analyst at research company Ovum. But such an arrangement is a nightmare when it comes to planning backup scenarios. “Any kind of disaster recovery based around multiple servers or multiple islands is very difficult,” explains Johnson.

“There are lots of organisations with a variety of storage solutions. How do they know their whole environment is being backed up?” asks Richard Collins, head of Computer Associates storage division.

Everyone agrees that widely dispersed servers and storage are a horror story waiting to happen. “If you have a lot of point solutions out there, it’s all too easy to have different policies on them. There needs to be some consistency,” says Grant Smith, storage manager at Tivoli. Greg Bowden, national business manager for integrator Dimension Data, agrees: “You can’t control a large infrastructure with point solutions all over the place.”

Nonetheless, for a variety of reasons, capacity demands continue to rise. “People are frightened to delete data,” according to Joan Tunstall, marketing manager for StorageTek. “It may have some business value that you don’t perceive today. If it’s something like e-mail, you may get into trouble for deleting it.” (Just ask the guys at Enron or Microsoft.)

So how can you save on personnel costs and work more efficiently? Make things simpler by consolidating. “Consolidation is something that has really become top-of-mind for many businesses,” says Christian. “It’s not only about server consolidation, but bringing everything together. It’s not about the upfront costs only.”

Others agree. “The umbrella trend is simplification and consolidation, bringing together islands of storage from servers and desktops,” says Johnson. “It’s a pattern that’s going to continue for some time.”

“If you consolidate into a central pool, you only need one lot of working space,” offers Titterington. “Of course you can just buy more disks, but then you have a big management problem on your hands. You get to the point where it takes longer to do the backup than there are hours in the day.”

Hard-headed planning is critical. Businesses are asking questions prior to authorising purchases of hardware or changes in infrastructure. “Lots of CIOs are requesting more information on the types of data that reside in the organisation,” says Collins. “We’ve also seen huge growth for running storage resource management services.”

There are limits to the process of consolidation, but even if dispersed hardware is unavoidable, it can often be better managed. “It may not be physically possible to consolidate all your storage, but you can at least unify your storage management,” says Smith.

Network it baby, yeah!

“Storage technology is in some ways a very immature market, because there’s still a high rate of innovation,” says Clive Gold, ANZ marketing director for EMC.

Yet that innovation is not always in areas of high customer demand. “Nobody’s knocking down our doors and saying they need a faster subsystem or more capacity,” said Gold.

The main area of development in storage hardware in recent years has been the move towards networked storage solutions. “Organisations are looking for a more holistic view of their storage assets, and are looking to connect islands of storage with new, lower-cost interconnect options,” says Simon Elisha, senior systems engineer for Veritas.

Overriding all of this is the need to make the management of all of this storage cheaper. The move to networking is seeing a decline in simple direct attached storage, though not to the point where it is becoming obsolete.

“I don’t see direct attached storage dying out, but people will be networking storage devices almost universally,” says Titterington. In Johnson’s view, “Direct attached storage still has a place, in test environments and in standalone storage.”

Another important market is in notebook users, who have unique backup needs.

Once committed to a networked system, the dominant debate in recent years has been over two approaches with disturbingly similar acronyms: storage area networks (SAN) and network attached storage (NAS). Storage area networks allow any-to-any connections between servers and back-end storage, blurring the distinction between individual devices, while NAS simply makes storage devices network accessible.

While vendors have promoted both technologies as rivals, the truth is a little less dramatic. “NAS versus SAN—that argument is going away,” said Christian.

In Elisha’s view, “SAN is by far the most effective solution for most enterprise storage requirements, particularly for database applications. NAS has a place as a file server on steroids, but the rush to host all applications on NAS devices is a little premature.” “SAN and NAS are potentially quite complimentary,” says Titterington. “SAN is a very bulk, heavy-duty, low-level type service. Once you’ve got it up and working, it’s robust, reliable, and can handle large volumes of data. It’s mostly used within a data centre or within one site. You’re looking at LAN scale rather than WAN scale. NAS works at the file level and can connect to any other network. You can just use Ethernet for it if you’re not doing particularly high volumes. And you could have a SAN behind it; NAS can be used as a gateway into the SAN.”

SANs are also working their way further down the enterprise food chain. “I don’t think company size plays a part in choosing a SAN—resources, infrastructure, and budget are the key indicators,” says Johnson. That doesn’t mean that either family of products are ready for true mass-market adoption. “The current products are not very friendly,” according to Titterington. “There’s a lack of interoperability. Most companies have lots of different equipment for historical reasons, but it tends to mean the first thing you need is an interoperability lab.”

Titterington estimates you can easily spend 10 percent of a SAN-related budget on trials and tests.

“NAS is a cheaper and easier option, but SANs offer better security,” says Alvin Ong, regional sales manager for Maxtor.

Not everyone is convinced. “The inbuilt security mechanisms for SANs are pretty crude,” says Titterington. “It’s supposed to limit via server zones, but it’s alleged that even that can be breached.”

“What we are seeing now is a convergence of the two with NAS heads built upon SAN backbones, enabling enterprises to service all areas of the business with appropriate technology,” says Elisha.

Whatever the network architecture, new standards for high-speed data transfer such as iSCSI and Serial ATA are essential to make real-time access to high-volume storage a reality. Titterington believes the technology that we need most seriously is iSCSI—Gigabit Ethernet is only the precursor. But both options may need further enhancement.

“iSCSI is still very immature,” says Ray Morcos, systems engineer at SGI.

“Serial ATA on its own is nothing special,” says Bert Noah, enterprise solutions group director for Acer. “The development of interoperability for serial ATA will prove its success.”

At the moment, the most reliable interconnect remains Fibre Channel. Its high speed has been matched by a high price, but that is changing. “The price gap between SCSI and Fibre Channel has almost disappeared,” said Noah, predicting the two technologies will be equal in cost by year’s end.

Yet arguments over technology shouldn’t blind IT managers to the bigger question of business processes. “It doesn’t matter how you connect into it. The big bang for buck comes when you can get a single way to backup,” says Gold.

Software is the key

“Software is something that people should look at first,” says Smith. And there is no doubt that software is attracting a lot of attention, because the attention goes where the pain is.

The key element of storage software is the ability to define company-wide policies that will remain consistent even as storage needs grow. “The idea of policy management is to allow growth in the environment without needing to add headcount all the time,” says Smith. “If you’re managing after the event, it takes so many more people.”

Storage needs to be seen as an infrastructure layer. “More and more customers are putting in a storage layer,” says Bowden. “You can build a disaster recovery plan based around your storage layer rather than 50 or 60 verticals.”

Simple software improvements have also enhanced basic functions such as backup, allowing, for instance, files to be backed up on the fly or databases to be backed up in copy-only mode.

Software can also deal more intelligently with backup routines and other data management tasks. “If during the day there were just a handful of changes made, why do you need to zero the database?” says Christian.

Yet backup shouldn’t be over-emphasised. “Often people just look at the backup side of the equation, but from the business side restore is more important,” warns Smith. “Software is starting to provide business functions, rather than just technical management stuff,” adds Bowden, citing the example of chargeback functions built into some management platforms.

The general shift to GUI-based policy management has also simplified storage management. The GUI doesn’t rule absolutely, though. “Those people who come from a Unix background still prefer the command line,” says Smith.

Software improvements can still be made. “We’re starting to see the ability to understand information, but storage management software still has some way to go,” according to Bowden. “Software tools alone are not the panacea.”

The bottom line: plan for your software needs, but don’t over-estimate what software can do for you. “The storage software is a tool,” says Smith. “It still comes back to understanding the processes and the procedures and integrating that into the organisation.”

Outsourcing?

Storage is often viewed as a prime candidate for outsourcing: it’s a fairly basic application with good economies of scale. But many storage industry watchers are sceptical about the value of businesses outsourcing their storage, or even their inclination to do so.

“People are still very paranoid about outsourcing what they hold near to their heart: their data,” says Christian. “We haven’t seen a big takeup in storage. You can give them your network and your desktops, because that’s all replaceable. But not storage. Business is too competitive, and there’s too much paranoia.”

“Outsourcing storage is still a niche area, as most organisations are not interested in losing control of the crown jewels,” says Elisha.

According to Titterington, it hasn’t taken off in a big way. “It is being done, but not in isolation. There aren’t many people who are buying it as a standalone service. Some people are outsourcing backup, but not the frontline storage needs.”

That may change in the future. “As the speed of bandwidth goes up and prices goes down, it’s becoming more realistic,” says Doug Oates, general manager for Pihana Pacific, which offers outsourced storage as part of its managed hosting facilities. “Most customers are looking for a solution that gives them guaranteed service levels. They can start small and scale with demand.”

Oates cautions against outsourcing purely as a cost-cutting activity. “It’s not just about money—it’s about getting availability of data.” Companies with dispersed offices in a single city, for instance, may find centralised storage more effective in providing speedy access to all locations.

“There’s still some early concerns about privacy and security of information,” says Bowden. “There’s some early resistance to outsourcing. It’s a case of the market being ready for it. Through partnerships, that area will grow over time.”

At the high end, resistance is likely to remain strong. “Our clients are the guys that really have gigantic amounts of data and they have to have reliable, consistent access,” said SGI’s Morcos. “Most of our clients would require that data to be in their own environment.” Outsourcing is still on people’s roadmaps as an option. “It’s no different than the outsourcing decision in general,” concludes Johnson. “So long as you have a set policy and the ability to check the results of your outsourcing, it comes down to a commercial decision,” adds Collins.

Value for money

Is it a prime consideration for storage purchases? Opinions differ. -I think vendors make a mistake by over-stressing ROI as such," says Titterington. -Storage is something people have to have. Companies want to provide it as cheaply as they can. It's very hard to take ROI into storage. The calculations would take you far longer than the value of the answer.

-The justification is largely on cost savings and then, when you get into backup, you're buying an insurance policy. People are putting forward ROI arguments based on data mining, but I don't think many are profiting from it yet."

Others disagree. -Storage is an investment. For any organisation to invest, they must find some kind of return," says Johnson. -If we're talking about simplification or ease of management, there are key TCO benefits."

According to Christian, -We really have been able to add value to organisations through consolidation. Maintaining all those servers costs a lot of money from a management perspective."

-If we can deliver the same services without having to backup everything, there's a lot of ROI benefits," says Smith. -It's in the interests of organisations to get a handle on their storage."

-ROI comes down to the question of suitabilityââ,¬"purchasing the right type of storage for certain environments," says Ong. -Customers need to invest in a storage solution that is manageable, scalable, and interoperable."

So what can you do to get better value? -The best way to get value is by getting the lowest possible initial buy-price, by using storage efficiently, and by streamlining the management aspect as much as possible," says Elisha.

Beyond the general rule of plan early and plan often, storage systems can help enhance specific activities.

For instance, faster storage options make it easier to test enterprise applications with close-to-current copies of production data, without disrupting normal operations.

-Using snapshotting or cloning, you can really reduce the time for testing things like ERP systems," says Bowden.

And with those earlier warnings about overall costs in mind, the relative cheapness of disk storage has had one important impact: reducing the cost and speed of backup.

-More and more, disk backup and recovery are taking the place of tape," says Christian. -You only need to go to tape for archiving." In the end, an argument about ROI from storage isn't going to mean you won't need to buy storage solutions.

-Storage is not an optional spend, it's not discretionary, it's essential," says Tunstall. -If you build a good infrastructure, it's going to deliver better value over the long term."

Bad planning means little chance of a value storage investment. -You can't get ROI if you don't know everything about your environment," says Collins. -Using technology to reallocate resources is a key driver."

But ultimately, storage remains a means to an end. -What is the enabler of the business?" asks Gold. -Storage isn't going to do it, but information is."

Storage: an executive summary

• Consolidate your storage. Outcrops of storage on isolated servers will quickly become a management nightmare.
• Network your storage. Connected storage is already the norm in most enterprises, and higher bandwidth technologies should convert the doubters.
• Get good management software. An efficient storage infrastructure and well-considered policies mean your staff will spend less time on routine tasks such as backup and more time on value-enhancing activities.
• Consider outsourcing, with care. Outsourcing can work well for storage applications, but only if you understand how it fits into your total organisational plan.

Subscribe now to Australian Technology & Business magazine.

Copyright © 2009 CBS Interactive, a CBS Company. All Rights Reserved.
ZDNET is a registered service mark of CBS Interactive. ZDNET Logo is a service mark of CBS Interactive.