How do you back-up 500 terabytes?
Gore says the problem with being a web host is rapidly changing data, with 40 per cent of the content Melbourne IT manages changing every seven days. This problem is magnified by the long lifetime of backups.
"We need to keep a backup of not only that content, but also those delta changes for as long as we keep backups for a standard customer. Typically that's up to six months," says Gore.
HD image by Clix, Royalty free
Melbourne IT's solution involved several forms of storage media. "What we have decided that we need to invest in — and we have been doing this for a number of years — is disk to disk to tape backup," says Gore.
This allowed the company to manage risk across a series of different storage media.
"The idea is that if we need to recover data, we can do it straight from fast disk, that's in the same datacentre," the IT architect says. "It's very rare for us to use backups for more than a month out. We still use tape to manage the risk of having a datacentre-wide outage."
Gore says Melbourne IT's tapes are encrypted and stored remotely by a third party. "To put it into perspective, we are spending in the order of more than $10,000 a month on tapes. It just comes down to the scale of managing all that data," he says.
The Future
Projecting future demand is difficult for any company, but Gore says the issue is magnified for hosting companies.
"As a hosting company, we are driven by what our customers are doing. A traditional enterprise would know where their business was going, and they would have three- and five-year plans. The IT [department] would know what they need to do to support that business," he says.
The biggest challenge for us, especially in Australia, is getting out hands on good quality datacentre space.
This challenge means creating highly dynamic IT environments, which can help with meeting future demands.
"As routine capacity planning we look at what we do every month in terms of business, and we forecast that out as a trend going forward and look at the growth on a month-by-month basis," Gore says.
"We then buy several months' worth of infrastructure in advance, and also keep several months' worth of infrastructure in the supply chain. We also manage a buffer, what we call the 'run rate head room', which is normally about 20 per cent reserved at any point in time."
However, Gore says continuing to scale presents unique challenges.
"The biggest challenge for us, especially in Australia, is getting out hands on good quality datacentre space... As soon as they build new datacentres they become full, and people are all trying to get out of datacentres that don't have the power density to meet today's IT requirements," he says.
This problem is compounded by the IT skills storage.
"[A problem] I think in 2009 will become even harder than datacentres is actually getting our hands on good quality IT staff which are experienced, have the right attitude, and can work in large, dynamic IT environments," the IT architect says.
Got an interesting story to tell about your company's IT environment? Drop ZDNet.com.au a line to have a chat.
4%
4%
Many very profitable businesses will purchase brand-name proprietary solutions for storage.
However, for a lot of others, getting large data storage is about bang per buck.
Good performance (high RPM) USB-connected 1TB Maxtor drives were on sale at Harvey Norman during Sept 08 for A$193.
A couple of those on networked PCs with auto-backup software and 'take-one-home' policy would do for many SMEs.
For details on software open source RAID setup, see http://www.tldp.org/HOWTO/Software-RAID-HOWTO.html and https://wiki.ubuntu.com/Raid for background.
For an open source Network Attached Server (NAS) which is a normal Linux PC network-attached server running RAID5, see www.freenas.org
EXT3 or NTFS file systems under RAID 5 (full redundancy) with spare drives already fitted is good workable storage, without any proprietary components. Moreover, you can then use simple CRON lines (at such and such an hour back up all files that changed on server X) with the simplicity of a single COPY command.