It wasn’t so long ago that a 1G disk drive made your computer the envy of every other nerd on the block. Now the tech enthusiast sees green when you have 4 to 10 TB. This is just at home!
Currently in the enterprise space we are seeing regular caches of data well in excess of 100TB. Oddly, the file systems holding this kind of data struggle when the files get much above 16TB in size. So we start carving our repositories into all sorts of folders. Sometimes this makes sense, but more often than not it is to get around the technology limitations of the storage subsystem being used.
The ideal storage system would be able to scale in terms of performance and capacity. This solution could mix and match SAS, SSD and near line SAS technologies so the right data is on the correct tier. Snapshots and replication, native, of course. The solution would allow us to create folders – but not be bound to them. We would be able to slide data from one folder or file system or from one protocol to another to another without having to reorganize the storage. There would be no taking losses due to repartitioning or any loss of performance due to hot spots. This storage would have internal data protection which doesn’t require RAID sets or aggregates that require bunches of parity drives or hot spares. More drives would just mean more storage efficiency. How awesome would it be to literally have 1000’s of drives working together for a single workload with petabytes of continuous space under your home drive or operations folder or development folder?
No crazy ideas here.
Not an odd desire.
Not a unique problem.
Yes, there are ways to build this “ideal” storage system described above yourself – if you have time for a serious science project.
But for the professional IT shop, who are looking for something that is fully supported, shrink wrapped, and ready to run – ENTER ISILON – stacks of nodes with lots of drives and a really smart OS, purpose built for serving (a lot of) files.
I have always seen the benefits of node based architectures, but also the downfall of the “backplane on a LAN” approach. Any kind of storage balancing or motion around the array could crush the network and then starve the end users coming over the same network, or in some cases; the same ports. Isilon sidesteps this common issue and threw an Infiniband network behind its nodes. Now we are talking about a backplane. The best part is that it is expandable as the number of ports on the switch. As the switch technology develops, so does the size and performance of the entire array.
OK, so we all don’t need to go this big, but what a great way to get there if you did. Crazy scalability, crazy performance, all shrink wrapped with support. I just have to figure out how to get one for my home theater…
Photo Credit: ChrisDag