What would Napster on steroids look like? Maybe like the futuristic distributed file system projects now in development at various universities and companies.
Microsoft Research is working on one such beast, a project code named Farsite. The University of California at Berkeley - with backing from the Defense Research Projects Agency, IBM and EMC - is working on another, code named OceanStore.
This week, in fact, the lead developer of Microsoft's Farsite is attending a research retreat sponsored by Berkeley computer science students working on OceanStore.
'Serverless storage'
These research projects are years away from finding their way into commercialised products --if, in fact, they ever do.
But they do indicate that companies and colleges are anticipating that in the not-too-distant future, bandwidth advances could expand exponentially the size and complexity of applications and services that users will expect to run over the Web.
A distributed file system controls where files are stored, for example, on people's own computers or on central servers. However, the central servers in charge of distributed file systems can get overtaxed when too many people use them.
Enter the concept of a "serverless file system"--or a kind of network of clients with no hierarchy, in the words Bill Bolosky, lead of Microsoft Research's Farsite project.
The 'serverless storage' era
"We know that (PC) disks aren't full today and machines are idle most of the time. Disks are getting bigger and faster than files. If you could build a centralised file server, it would be more reliable and it wouldn't require centralised administration," Bolosky explained.
Napster and other peer-to-peer (P2P) file-sharing software are built on a similar premise, but with one key difference, Bolosky said.
"Napster does have a central directory. Only its file store is peer-to-peer," he said.
With Farsite - a concept Bolosky and his five-person team has been dabbling with for the past year - all elements would be distributed. As such, as Bolosky and his colleagues have conceived of it, Farsite would be based on a true co-operative storage model, rather than a centralised or local one.
Power users only These kind of mega file systems aren't for the casual user.
Microsoft's target for Farsite, Bolosky said, is a "large company or university, meaning an organisation with around 105 machines, storing around 1010 files, containing around 1016 bytes of data."
You read that right: 100,000 computers networked together.
The OceanStore team has set its sights even higher, said Bolosky, talking about serverless networks consisting of 10 billion computers, containing 1023 bytes of data.
OceanStore, as it has been described on various Web sites, is more of a generalised object store than a pure file system.
But as it is being developed by the same Berkeley group that worked on the xFS file system, it has its roots in the distributed file system world.
And like Farsite, OceanStore has implications for mobile computing, as well.
One OceanStore Web site explains the concept this way: "Unlike other file systems, OceanStore provides truly nomadic data that is free to migrate and be replicated anywhere in the world."
