How big is your company? If you have hundreds of employees with desktop computers, you could put extra Tb disks in them and work out a redundant distributed filesystem.
Still going to cost a lot for disks and a lot of do-it-yourself setup, but piggy backs on existing network and power infrastructure, no need for expensive datacenter kit or new machines/tape drives/libraries/etc. and you could build into your existing desktop maintenance and renewal procedures.
I actually worked on a system to do this back in the late 90s code-named "Dark Iron". Our boss, who conceived of it, thought he was some kind of old school mainframe ("Big Iron") guy who talked a lot about using the unused capacity on all the desktops at companies at night (the "Dark" part, like "dark fiber").
I wrote the Linux kernel block device module and the other guy on the project wrote the part that ran on all the users desktops, mainly just a key-value store indexed by block number (the users liked to kill the process, which made it terribly unreliable). Back then, we used Linux purely for development, under the theory that once we "proved" it worked with the rinkydink Linux open source operating system, the company could then justify the purchase of the HP-UX and AIX kernel development kits and we'd port it to machines that people "actually" use.
We all know how that last part turned out.
But really, if you want to do something exactly like this (rather than the VMs that davecampbell suggests) these days (that is, since about 2003), you'd be best off using md software raid over Network Block Device. I believe there are Network Block Device storage ends (is that the client or the server?) for platforms other than Linux. Even a simplistic AOE service would do.
Your first paragraph suggests you look down on the idea - do you?
How far did you get and what happened to it? I wasn't suggesting a VM based idea, more of a daemon/windows service.
Having said that, there's no real need for it to be a filesystem per-se, it could be any kind of store with a central interface (web based, for instance) and then it wouldn't have to deal with parts of a filesystem vanishing arbitrarily. As long as the data didn't need to be live all the time, that is. It could even hook into Wake-On-LAN, and also benefit from any power savings if machines are switched off overnight.
Your first paragraph suggests you look down on the idea - do you?
It was grossly ahead of its time (remember, this was the late 90s). The assumption was that the network was faster than the disk, which in our implementation was never true because it required a reliable backing store, so it was network latency + disk desktop disk latency. And we couldn't use a lot of RAM as cache because desktops didn't have multi-gigabyte memory at the time -- even servers didn't.
How far did you get and what happened to it?
I assume it's on some backup tapes somewhere. Every so often I come across an old CDROM I made during the era and get this idea that the source code is buried on it somewhere. It was tied pretty closely to early 2.0 Linux kernels and the block layer that existed at the time (I don't know how much would be portable to modern kernels).
A certain company that sued Microsoft for a patent related to embedding executable content in web pages, that we had a working relationship with at the time, had claims on their web page that they owned the trademark for it, but as far as I know, we were the only ones to produce working code.
Both of us who were working on it eventually left for greener pastures (and that was like 12 years ago at this point).
I wasn't suggesting a VM based idea, more of a daemon/windows service.
Nah, davecampbell in this thread suggested the use of VMs. Using a VM might make deployment easier. There wasn't anything like vmware player back when I worked on this, and we had to go around to everyone's machine and install the new versions after hours with every release.
there's no real need for it to be a filesystem per-se, it could be any kind of store with a central interface (web based, for instance) and then it wouldn't have to deal with parts of a filesystem vanishing arbitrarily.
True. And with removing that constraint/use-case, there are a number of things today that already provide this: Amazon S3 is one. We implemented it as a generic block device and put a filesystem on top of that, mainly for testing. One of the "goals" was to eventually use the block device as an Oracle tablespace file (once ported to HP-UX, since Oracle didn't exist on Linux at the time, or had barely come out for Linux).
Like I said, if you want to do this today, try it with something like md on top of NBD. Although, see the section "Late news - intelligent mirroring" on http://enbd.sourceforge.net/ (which is itself old).
But today, I wouldn't even bother. I'd just bite the bullet and buy the disks (or use some other system that is more directly under sysadmin control than employee's desktops). I think there's way too much variability in using desktop clients that were deployed for another use (desktop use) for this purpose.
interesting idea - i wonder if a hadoop cluster with nodes on virtual machines installed on employee desktops make sense? seems like a lot of work to install extra drives on all machines but deploying the virtual machines as nodes might be doable. the distributed drives could have the VM installed right on it. i like the idea.
Still going to cost a lot for disks and a lot of do-it-yourself setup, but piggy backs on existing network and power infrastructure, no need for expensive datacenter kit or new machines/tape drives/libraries/etc. and you could build into your existing desktop maintenance and renewal procedures.
Do you want a hacky build-it-yourself solution?