Bureau forecasts AU$14m data store

The Australian Bureau of Meteorology will soon start building a AU$14 million storage facility to satisfy its expanding data requirements for the next five years.

The agency's role is to observe and understand Australia's weather and climate, providing services to support the nation's needs in the area. It is headquartered in Melbourne, with some 1,420 staff located in Australia and internationally, according to its 2004/05 annual report.

As with many research bodies, the Bureau of Meteorology has traditionally been a heavy user of both computational resources and data storage. For example, in 2001 the group acknowledged it was often a matter of trying to keep its ahead above water when it came to storage. The bureau's current tiered storage system stores some 480 terabytes of data in 32 million files.

Its latest effort to stay ahead of that game was revealed this week in an effort the bureau dubbed its "Large Scale Data Storage System (LSDSS)". In tender documents, the agency said it wanted to acquire an "integrated and highly automated data storage and data management system" that would enable it to migrate from a machine-specific storage architecture to a data-centric storage architecture.

"The LSDSS will be an integrated system including hardware, fabric and network, software, virtualisation capabilities, licences, professional services including training for Bureau staff, and system support and maintenance services," the bureau wrote.

"It will provide foundation data storage services for Bureau operational research programs from October 2007 through 1 October 2012, and will form the basis for data services and evolutionary upgrades beyond that time."

The new system will store a large amount of scientific data kept in the netCDF format -- a standard common to the meteorological community -- as well as more mundane information in, for example, Oracle databases and Microsoft Exchange mail servers.

The Bureau of Meteorology is proposing to spend a total of AU$14 million on the system, roughly averaged out over five years. The agency will pick a supplier in early September this year, with initial components of the LSDSS to become operational in early November.

The bureau currently makes use of a multitude of vendors in its storage and computational infrastructure, for example Sun Microsystems (and its subsidiary StorageTek), IBM, NEC, Hewlett-Packard and Dell.

The Bureau of Meteorology's new LSDSS system will be one of the nation's largest storage projects, sitting alongside efforts such as CSIRO's recent AU$4 million deal with Hitachi and Volante to boost its own capacity to three petabytes (three thousand terabytes).