Digital Storage Observations

Image by Lyn Millett via Flickr

Technology Thursday

I had an abbreviated exchange with Dr. Jen Gunter on Twitter the other day about hard drives and storage. I promised to write up a few notes about how to think about disk storage needs. One reason concerns about storage are popping up for typical users is the increased availability and use of solid state (SSD) hard drives. These are the drives contained in the Macbook Air (which Dr. Gunter was discussing.) These drives are much, much faster than traditional hard drives, but also more expensive per unit of storage. For example, right now at Best Buy a 320 Gigabyte laptop hard drive (not SSD) is on sale for $45, whereas a 120 gigabyte SSD laptop hard drive is going for $250.

But even setting aside SSD drives, I found myself bumping up against the disk space limits in my iMac this year and just recently replaced the 300 gigabyte internal drive with a 1000 gigabyte (1 terabyte) drive. So what's the deal with storage and how should one think about it? I have some thoughts - but of course one size does not fit all. (On re-read, that's a bad pun, but I'm leaving it in.) Nevertheless, here are some things to consider.

For typical users there are generally two kinds of 'storage' or 'memory' that matter. There's the hard drive (permanent) storage and RAM (often referred to as 'memory') which is volatile storage (and goes away if you shut down the computer or the power goes out). Generally speaking, more of each is better. RAM is more expensive (and there's much less of it in the machine), but it is key to how fast and responsive your computer feels while using it. You might hear references to memory-intensive software applications (video and photo and some gaming applications in particular tend to stress RAM) and if you tend to use those, then definitely try to get as much RAM as your computer will allow. Hard drives (permanent storage) are where all your files, data, software applications, and system software live.

For casual users, the amount of hard drive space that comes with your computer is probably sufficient. However, it can quickly become insufficient if you start collecting and keeping a lot of data (say, in the form of videos, photos, podcasts, music, and so on) or if you are on a long replacement cycle (more than 3-4 years, I'd say) for your computer. Or if you've got a computer with an SSD drive (and thus, probably, less storage than comparable computers with traditional drives.) In my case, I bought my iMac in 2007, so it's unsurprising to me that the original hard drive is no longer sufficient. (With the upgrade, I hope I can avoid replacing it for another 2-3 years; I maxed out the RAM this time, too.)

So, if you're feeling short on space for whatever reason, what are your options? Some ideas:

• First, do a quick check and see if you have a bunch of stuff you don't need or want that's taking up a lot of space. Did you inadvertently download 50G worth of video podcasts? Do you have a bunch of ripped DVDs that you're never going to watch again? But, I say to do this check quickly because in my view, time spent culling your hard drive is often wasted time. Disk storage is ever cheaper (even SSD storage) and generally speaking you're better off just paying a bit for more storage than spending oodles of time trolling through your hard drive for files to delete. But sometimes there'll be an obvious big cluster of things taking up space that you don't actually need. There are some tools that provide a nice graphical way to analyze the data on your drive to see what's taking up the most room. GrandPerspective for the Mac and WinDirStat for Windows.
• Second, consider using external storage for some data. This is a tough call. External hard drives are cheaper than hard drives that need to be installed inside your computer (desktop or laptop). But they take up space on your desk, have to be plugged in to your computer to be used (taking up a USB port or other connector), tend to be slower than internal drives, and are one more thing to maintain (and backup - see below). But they are appropriate for some things. In our house, our iTunes library is kept on an external hard drive. iTunes isn't essential to my day to day efforts, and any data I manipulate there is very lightweight (I add material to it now and then, and I update and adjust playlists) so the comparative slowness is not an issue. Conversely, I keep my main photo library/database on my internal drive, since photo manipulation is already sometimes slow and if I'm editing lots of photos, I want things to be as snappy as possible.
• Another option is to use network/online/cloud storage for some things. For this, you need to be sure to have internet access. And it's typically even slower than an external drive. But keeping things online can be useful. I've written before about how I use Evernote and Dropbox to do some of this. I keep recipes in Evernote, for example. And I keep a few small backups of important files on Dropbox, as well as using it to transfer files between machines. I also "store" certain things in gmail, such as all package tracking information (which is temporary, after all).
• Most importantly, though, is to move towards a reliable backup regimen. DIsk drives don't fail often, but when they do (in my experience), it tends to be a catastrophic failure. Work out some sort of backup routine and automate as much of it as possible. For my Macs, I use Time Machine and Super Duper. I make weekly backups of my photo database and monthly offsite backups of our Quicken data. What I haven't gotten rigorous about yet, though, is backups of what I'm keeping online (because online services, especially those that are free, could just close up shop at any moment). I download gmail to my iMac daily, but I need to start exporting Evernote and my Pinboard bookmarks (soon I'll try to write up a post about the recent Delicious fiasco) on a regular basis.

There are of course other approaches to managing digital data and storage - these are just some quick thoughts. At a high level though, I think the issue is to decide what kinds of data you have that need storage, how much, how critical is it, and how regularly do you need to access or manipulate it - answers to those questions will help determine what kind of storage and backup regimens to put in place.

So, there are some thoughts about storage. It's a somewhat complicated business. But the more of our information that goes digital or is born digital, the more complicated it's bound to be. People are complicated. Our data is complicated. On the bright side, once a robust set of backups and redundancies is in place, one can breathe a bit easier. Maybe. I don't know yet, because I'm still not to the point where I feel like my backup routines are in a good place. But, I'm making progress!

Comments

[Andrew]

My backup routine quickly became insufficient when we got an HD camcorder. Now that I take 1080 HD video of our little boo, terabyte drives don't see so big anymore (it's roughly 8 GB per hour of video). And there's no way you're moving that much data to the cloud for backup with the pitiful upload speeds of home broadband connections. So my off-site backup is actually a hard drive in a safe-deposit box at our bank around the corner that I update a few times a year.

[Kristie Taylor]

This is tangential to your post -- but how does storing recipes in Evernote work for you? I've tried doing that, but I find it very inconvenient to have the laptop in the kitchen with me. Maybe someday if I move to a tablet computer, but for now, I can't make it work. I open to ideas though, because I'd really like to do that.

[Lyn]

Kristie: iPhone.

And I sometimes set the laptop on the dining table and just go glance at it now and then to remind myself what's next.

I also print out more complicated recipes sometimes for one-time use. I don't care if something spills on it.

I find any of these options easier than worrying about a cookbook in the kitchen.